===============================================================================
       Juno Waves Ground Support Software - File Formats and Storage
     

First Rev: 2008-02-26 through 2008-12-03, Chris Piker
                    
Revision History
----------------
2009-10-22  C. Piker      Changed file naming patterns to keep low rate and
                          high rate data seperate.  
                          
2010-02-11  C. Piker      Added section 4.3.2.3 which defines the "JPL FDM file
                          Reader Processing Block"

2010-02-17  C. Piker      Added section 4.

2010-02-25  C. Piker      Alterations for adding I/Q channel recombination

2010-04-20  C. Piker      Added MRO description block documentation
                
===============================================================================



1. DATA LEVEL DEFINITIONS
=========================
Stream - Raw Stream from test equipment without modifications.  Often these
         may not be saved.

Raw  (Level 1)
       - Packetized stream, see Waves Ground Support Raw Packet header
         description below.  In this format headers that will not be
         down linked in flight are removed.  These are usually saved as files.
         There are three types of raw files, science data (LRS or HRS),
         Housekeeping data (HSK), and other data (XX) which includes 
         response data (whatever that is).

Uncalibrated (Level 2)
       - Packetized data that is unsegmented, decompressed and unpacked. See
         Waves uncalibrated packet header description below.

PDS (Level 3)
       - Planetary Data Systems archive approved formats.  These are defined
         in waves_pds_fmts_stor.txt

Text   - Documents explaining the purpose for collecting a stream as well as
         the current instrument software load may be generated as well.


2. FILE NAME PATTERNS
=====================
To bring some order to the plethora of files generated in testing the Waves
instrument the following file naming guidelines are used by the ground support
software.

   File Base Names
   ---------------
   Test data files do not have a predetermined length.  Usually they are 
   collected for the purposes of a particular test and accumulate data until
   the receiving software is shut off.  Thus base file names follow the
   pattern:
   
     YYYY_MM_DDThh_mm_ss
   
   where YYYY is the year, MM is the month DD is the day of month, hh is the
   hour on a 24 hour clock, mm is the minute and ss is the second.  File names
   using this format are legal on both Windows and Unix.

   
   File Suffixes by Data Level
   ---------------------------
   stream - Often the initial stream is repackaged and output in Raw format, 
            However if these are saved it is with the suffix ".strm".

   raw    - The re-packetized streams will be saved with a two part suffix 
            that indicates the data kind and data format.  These are:
            
            *.sci.raw - Waves Low Rate and High Rate Science
               Note, this is CIP no. 8 (was 4) for low rate and for high
               rate there are no CIP packets.
               
            *.hsk.raw - Waves Housekeeping, on low rate this is CIP no. 6 (was 3)
            
            *.XX.raw - Other kind, where is XX is most likely the CIP data
                       type integer if packed up on LRS output.
            
   ucal   - The final basic processing step is to produce unsegmented, 
            decompressed but as yet uncalibrated data files.  However, other
            steps along that path may be in this directory.  
   
            *.upk - Unpacked data, this is the first stage to contain a 
                    defined Science Data Staus block, which is used for all
                    following processing.
                    
            *.useg - Contains unsegmented mini-packets, that may or may not
                     still be compressed.
                     
            *.dcmp - Decompressed mini-packets, may still be glopped.
   
   PKT    - Unsegmented, decompressed, uncalibrated files of a fixed coverage
            period as taken from the encoded SCLK end in *.PKT 

   DAT    - Reformated PDS data products.
   

3. STORAGE HIERARCHY
====================
The following storage hierarchy will be used for automatic file placement 
underneath some user-specified top level directory.

Note: Times indicated below are in UTC.

Top
 |
 +--YYYY_MM                              Year and month directory, may be
    |                                    derived from workstation time, or
    |                                    once it is reliable, from SCLK fields
    |
    +--DAY_OF_MONTH                      Individual data files, at most
       |                                 one file is generated per data type
       |                                 for each run of the waves_sce_reader.
       |
       +--YYYY_MM_DDThh_mm_ss.cmd.raw    Raw command bytes as transmitted to
       |                                 the SCE bin_cmd block.          
       |
       +--YYYY_MM_DDThh_mm_ss.lrs.strm   Unaltered bytes as received from the
       |                                 Lockheed Martin spacecraft emulator's
       |                                 low speed interface.
       |
       +--YYYY_MM_DDThh_mm_ss.hrs.strm   Unaltered bytes as received from the
       |                                 Lockheed Martin spacecraft emulator's
       |                                 high speed interface.
       |
       +--YYYY_MM_DDThh.lrs.raw          Raw Science data packets primarily
       |                                 originating with the instrument's low
       |                                 rate science interface.  These may
       |                                 be collected from a variety of 
       |                                 interfaces.  Data section is raw 
       |                                 Waves IDP data.
       |
       +--YYYY_MM_DDThh.hrs_rec.raw      Raw Science data packets primarily
       |                                 originating with the instrument's high
       |                                 rate science interface.  These may
       |                                 be collected from a variety of 
       |                                 interfaces.  Data section is raw 
       |                                 Waves IDP data.
       |
       +--YYYY_MM_DDThh.hsk.raw          Raw Housekeeping data packets.  Data
       |                                 is raw Waves IDP data.  These originate
       |                                 from the instrument's low rate science
       |                                 interface, but are seperated out from
       |                                 the science date.
       |
       +--YYYY_MM_DDThh_mm_ss.XX.raw     For packets that are not known to 
       |                                 be science or hsk, where XX is some
       |                                 identifier, probably the CIP type
       |                                 integer if available.
       |
       +--YYYY_MM_DDThh_mm_ss.lrs.upk    Unpacked Low Rate Science data.  There
       |                                 exists one minipkt's data in each GSE
       |                                 packet.  This may be only part of a
       |                                 data set (capture).  Data may be
       |                                 segmented, compressed and or glopped.
       |
       +--YYYY_MM_DDThh_mm_ss.hrs.upk    Unpacked High Rate Science data.  
       |                                 Data is in the same format as LRS.
       |
       +--YYYY_MM_DDThh_mm_ss.lrs.useg   Unsegmented Low Rate Science data. 
       |                                 There exists one or more complete data
       |                                 sets (captures) in each GSE packet.
       |                                 This data may be compressed.  If there
       |                                 is more than one capture in a GSE
       |                                 packet the data is glopped.
       |
       +--YYYY_MM_DDThh_mm_ss.hrs.useg   Unsegmented Low Rate Science data. 
       |                                 Data is in the same format as LRS.
       |
       +--YYYY_MM_DDThh_mm_ss.lrs.dcmp   Decompressed Low Rate Science data.
       |                                 There exists one or more complete data
       |                                 sets (captures) in each GSE packet. 
       |                                 If there is more than one capture in
       |                                 a GSE packet that packet is glopped.
       | 
       +--YYYY_MM_DDThh_mm_ss.hrs.dcmp   Decompressed High Rate Science data.
       |                                 Data is in the same format as LRS.
       |
       +--YYYY_MM_DDThh_mm_ss.lrs.ucal   Uncalibrated Low Rate Science data.
       |                                 There is one and only one data set
       |                                 (capture) per GSE packet.  Data is not 
       |                                 glopped, compressed, segmented, or
       |                                 packed
       |
       +--YYYY_MM_DDThh_mm_ss.hrs.ucal   Uncalibrated High Rate Science data.
                                         Data is in the same format as LRS.
                    

3.1 Times Used for File and Directory Names
-------------------------------------------
For all data in the STREAM, RAW, and UCAL trees, times are taken form the
workstation time of the original data collection and files cover an arbitrary
duration.


3.2 Processing Chain
--------------------
The storage hierarchy above is listed in dependency order not alphabetical
order, the basic processing steps, in order, are:

  1. Command data are sent to waves and optionally copied under CMD.

  2. Data are read from some source and packetized.  The packetized data is
     placed under RAW.  Optionally a copy of the input is placed under STREAM.

  3. Data are read from RAW and after running through a series of programs goes
     under UCAL.  

  4. Data are sorted, but not altered, into Level 2 product in the PDS hierarcy
     (see waves_pds_fmts_stor.txt)
  
  5. Data are reformated, calibrated and stored as level 3 products in the PDS
     hierarcy (see waves_pds_fmts_stor.txt)


4 GROUND SUPPORT DATA FORMAT
============================
The format for data supplied from Waves to it's support systems is called the 
IDP (Instrument Dependent Protocol).  As development proceeds, IDP packets will
be supplied embedded in various headers.  For example the Lockheed Martin
Spacecraft Emulators (SCEs) provide low rate data as IP packets, wrapping UDP
packets, wrapping CIP packets, which contain the actual waves IDP data.

No matter the source of the data it is the task of the various processors to
output data streams in a standardized form.  This form is:

   1. A General Ground Support Packet Header.
   
   2. Instrument and Mission specific Headers.
   
   3. Instrument data.
   
   4. A copy of the total packet length.


4.1 Ground Processing Intermediate Steps and Data Contents
----------------------------------------------------------
The input data for ground processing is the Waves output stream along with any
headers assigned by "out of house" spacecraft or mission operations data
handling systems.  One of the main tasks of the ground processing software is
to collect data from an upstream source, for instance the Juno spacecraft
emulator, and to produce standardize uncalibrated data whose format is
independent of how the data was packaged for delivery by the Waves instrument.  

Waves packages data in a variety of ways in order to meet restrictions on packet
lengths and to efficiently utilize the available telemetry bandwidth.  Most of
the in-flight packaging must be undone on the ground to recover the collected
waveforms and spectra.   

Each ground packet consists of a header and a data section.  Initially the data
section is quite complicated, containing one IDP packet as transmitted from
Waves and thus is a non-trivial mixture of headers and in-flight packaged data.
Ground software then undoes most of the in-flight packaging.  At the point 
that uncalibrated data sets are generated by the ground software, the data 
section consists of only one waveform or one spectra in a simply readable
format.   

The following table lists the ground processing steps leading up to uncalibrated
data sets along with the data section contents after each step.  The routine
or program at each step attempts to output data packets in the Uncalibrated
state, i.e. one decompressed capture per packet.  Note that only science data
proceeds through all the steps.  Housekeeping data and other CIP types stop
after the reading stage.


  Step              Science Data Section Contents
  --------------    --------------------------------------
  Reading           1 Waves IDP containing 1-N mini-packets (HSK Stops Here)
  Unpacking         1 Mini-packet Data -or- 1 capture 
  Unsegmentation    1 Mini-packet Data -or- 1 capture 
  Decompression     1 Mini-packet Data -or- 1 capture 
  Deglopping        1 capture

The program or routine that outputs data in the Uncalibrated state assures that
all temporary fields in the Science Data Status Block (see 4.3.3.1) are zeroed
and sets the Data Contents Indicator (see 4.3.1) to 0x4 (1 Capture).

 
4.2 Ground Support Headers (GSE Header)
---------------------------------------
Juno Waves ground support headers follow a general pattern that is reusable 
for other instruments.  

    +--------------------+
    |       Prefix       |
    +--------------------+
    |   Header Block 1   |
    +--------------------+
    |   Header Block 2   |
    +--------------------+
    ...
    +--------------------+
    |   Header Block N   |
    +--------------------+
    |  Optional Padding  |
    +--------------------+
    
The first 16 bytes of the header, called the prefix, provides a 32-bit sync 
pattern, a CRC, as well as the lengths of the header and data sections of the
packet.  It also contains the spacecraft ID which acts as as name space
indicator for various enumerations.   The prefix is followed by a series of
information blocks some of which are general and some of which are instrument
specific.

Padding consists of zero bytes, 0x00, and where possible 0x00 should never
indicate a filled in value.  For Waves, all headers must be padded out such 
that they are 256 bytes long.


4.2.1 Prefix
--------------
There is one prefix per packet, it is always 16 bytes long.  The first 4 bytes
are the sync pattern 0xFA6C2741.  This is useful in the case that a file
becomes fragmented or has invalid length information for one or more packets.
          
       +--------+--------+--------+--------+--------+--------+--------+--------+
000-007|  0xFA  |  0x6C  |  0x27  |  0x41  |       CRC       |  HEAD_TAIL_LEN  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
008-015|        TOTAL_PACKET_LENGTH        |  SPACECRAFT_ID  | HDR_VER|  CONT  |
       +--------+--------+--------+--------+--------+--------+--------+--------+

CRC:
    16-bit unsigned MSB first integer containing the CRC-CCITT of everything
    that follows for this packet up to and including the trailing length bytes.
    See Appendix B for a definition of the CRC-CCITT algorithm in C.
  
HEAD_TAIL_LEN:
    A signed 16 bit MSB first integer containing the number of bytes in
    the GSE header alone this includes the sync pattern and the trailing
    4 length bytes.
    
    NOTICE:  In general GSE Headers are not required to be fixed length and
    are not required to have any special minimum length above the 16 bytes 
    required for the prefix.  However, since zero fill bytes are ignored at
    the end of the header, and to aid in debugging broken files, all Waves 
    headers are REQUIRED to be have MINIMUM of 260 bytes for this field.
    This means there are a minimum of 256 bytes in the header plus 4 bytes
    for the trailing length. 

TOTAL_PACKET_LENGTH:
    An unsigned 32bit MSB first integer containing the number of bytes in
    the total packet, including all data and all header and trailer bytes.
    
SPACECRAFT_ID:
    A 16-bit MSB first signed integer. If available this is the NAIF spacecraft
    code.  For Juno this is -61 or 0xFF 0xC3 as hex bytes.  Note that the 
    SPACECRAFT_ID sets the name space or "code table" for block types, 
    program IDs, and the CONTent byte.
    
HDR_VER:   
    An unsigned byte.  This document describes waves ground support packet
    version 0x04.
 
CONT:
   An identifier for the contents of the packet.
   
   For Juno this byte consists of two parts, the upper 4 bits are the Data
   Kind Indicator, this is the same as the CIP ID, even if the data was
   gathered from a source with out CIP headers.  Thus from:  
   
     documents/trunk/Waves_IDC_11Juno08-1.doc
   
      0x6 - Down-link Data (Housekeeping)
      0x7 - Process Data (Info from waves to Juno CDS)
      0x8 - Science Data
      
      0xE - MRO Data (Internally generated indicator, not from ext. ICD)
   
   The lower 4 bits are the Data Contents Indicator, the values for 
   science data are:
      
      0x1 - Raw IDP
      0x2 - 1 Mini-packet
      0x3 - 1 Mini-packet's Data
      0x4 - 1 A/D Capture (or one contiguous memory readout)
      0x5 - 1 Measurment


4.2.2 General Organization of Header Blocks 
-------------------------------------------
The rest of the header is series of Header Blocks, each of which may be up to
256 bytes in size and always start with a size byte and a block ID byte.  At a
minimum, this provides sufficient information for programs to skip over blocks
that are unrecognized.   

     Block    Process         
    Type ID   Type ID            Purpose
    -------   -------  ---------------------------------------------
      0x00       -     Never a legal block type
   
      0x10       -     Science data status
		0x11       -     Memory Read Out definition
		0x1A       -     High Rate Science Bin Header
		0x20       -     Wrapped CCSDS Header Applied on the spacecraft
   
      0x70      0x10   Genecic reader processing information,
      0x70      0x11   Lockheed Martin SCE Low rate reader processing info.
      0x70      0x12   Lockheed Martin SCE High rate reader processing info.
      
      0x70      0x20   Unpacker processing information
      0x70      0x30   Unsegmenter processing information
      0x70      0x40   Decompressor processing information
      0x70      0x50   Deglopper processing information
      0x70      0x60   Reformatter processing information
      0x70      0x70   Calibrator processing information
   
      0xA0       -     Program Error, Warning or Notice Number
      0xA7       -     Program Error, Warning or Notice Text Message
   
      0xD0       -     Aux. Information.
    
All programs shall output blocks into the header in ascending order by Block 
Type ID and then by Process Type ID if available.  This fixes block locations
within the header so that programs which only deal with a subset of all
possible content types may access various fields merely by absolute offset from
the start of the header.


4.2.3 Common Block Bytes
---------------------------
All blocks start with the following two bytes:

        +--------+--------+
r00-r01 |BLK_TYPE| BLK_LEN|
        +--------+--------+
  
BLK_TYPE - The Block Type ID.
  
BLK_LEN - An unsigned byte containing the number of bytes in the block
     minus one, thus the maximum block size is 256 bytes.
     

4.2.4 Common Processing Block Bytes
-----------------------------------
Processing blocks allow for storage of extra information associated with a 
particular data processing step.  All blocks with type id 0x70 start as follows:

       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  | BLK_LEN| PROCESS| PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r15|BEG_TM_US_DIV_100| ERRORS |WARNINGS|
       +--------+--------+--------+--------+

PROCESS
   An unsigned byte containing the identifier for the processing step.  Process
   IDs are spacecraft specific.
   
PROGRAM
   An unsigned byte containing the identifier for the program that handled the
   processing step.  See Appendix C for a list of program IDs.

BEGIN_TIME_SEC
   Unsigned MSB first 32-bit integer containing the time the process
   identified by the PROCESS byte was started.  It is the number of seconds
   elapsed since 00:00:00 on January 1, 1970, Coordinated Universal Time (UTC). 

BEG_TM_US_DIV_100 (Begin Microseconds Divided by 100):
   Unsigned MSB first 16-bit twos compliment integer containing 1/100th the
   number of microseconds since the begin time seconds.  To get the begin
   time microseconds, multiply this value by 100.  Thus the time resolution is
   0.1 milliseconds.

ERRORS
   The number of errors encountered during this processing step.  Associated
   error number or error message blocks should be included later in the header
   to provide error details.  

WARNINGS
   The number of warnings encountered during this processing step.  Associated
   warning number or warning message blocks should be included later in the
   header to provide warning details. 
   
   
4.2.5 Program Message Number Block
------------------------------------
TBD, used to store an error, warning, or notice number from a program.


4.2.6 Program Message Block
-----------------------------
TBD, used to store an error, warning, or notice message from a program.



4.3 Specific Block Definitions for Juno Waves Processing
--------------------------------------------------------
Future blocks maybe defined as needed.


4.3.1 Non-processing blocks
---------------------------
These header blocks do not represent the action of any particular program.  But
rather the packet as a whole.


4.3.1.1 Science Data Status Block
---------------------------------
The program handling mini-packet unpacking duties generates this block when
science data mini-packets are encountered in it's input stream.  The Science
Data Block defines the state of the data section and contains both permanent
fields that are retained after all processing is complete up to the Uncalibrated
state and temporary fields required for intermediate processing states.  This
block has the lowest type ID so as to always occur directly following the
prefix and thus its offsets are listed as absolute byte positions from the
start of the header.

Note that the Data Contents Indicator shall be 0x3 (1 Mini-packet's Data) or
0x4 (1 Capture) when this block is present. 

This block acts an expanded version of the mini-packet header.  Ideally the 
mini-packet header emitted by Waves would be reused for ground processing.  The
limitation in doing so is that the mini-packet headers transmitted from waves
are designed to be as small as possible and thus cannot describe reassembled
data.  Instead of including the mini-packet header as transmitted and then
ignoring some of it's fields, this data structure is defined so as to hold all
data states from initial unpacking of mini-packets up to calibrated captures.
See Appendix A: Waves IDP for information on data as transmitted from waves.


Block Size:  56 bytes
Block Start: Offset 16 from the start of the GSE header.

       +--------+--------+--------+--------+--------+--------+--------+--------+
000-007|  0x10  |  0x37  |MODIFIER| TLM_SRC|           COLLECT_SCLK            |
       +--------+--------+--------+--------+--------+--------+--------+--------+
008-015| COL_RTI|   AVG  |    HI_SEQ_NO    |          HI_REPORT_SCLK           |
       +--------+--------+--------+--------+--------+--------+--------+--------+
016-023|     IDP_CRC     |    LO_SEQ_NO    |          LO_REPORT_SCLK           |
       +--------+--------+--------+--------+--------+--------+--------+--------+
024-031|  MPH0  |  MPH1  |  MPH2  |  MPH3  |  MPH4  |  MPH5  |  MPH6  |  MPH7  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
032-039|  MS_G0 |  MS_G1 |  MS_G2 |  MS_G3 |  MS_G4 |  MS_G5 |  MS_G6 |  MS_G7 |
       +--------+--------+--------+--------+--------+--------+--------+--------+
040-047|  MS_G8 |  MS_G9 | MS_G10 | MS_G11 | MS_G12 | MS_G13 | MS_G14 | MS_G15 |
       +--------+--------+--------+--------+--------+--------+--------+--------+
048-055| MS_G16 | MS_G17 | MS_G18 | MS_G19 | MS_G20 | MS_G21 | MS_G22 |  FLAGS |
       +--------+--------+--------+--------+--------+--------+--------+--------+


MODIFIER:
   The ID of the last processing step to modify this block.

TLM_SRC
   A two part byte.  The *upper* 4-bits contain the *lowest* numeric telemetry
   source of the Waves IDP packets that contained this capture's data.  The
   *lower* 4 bits contain the *highest* numeric telemetry source of the Waves
   IDP packets that contained this capture's data.  The telemetry sources are:
   
       0x0 - Error
       0x1 - High Rate Science
       0x2 - Low Rate Science
       0x3 - Housekeeping
       
   Nominally all data for a single capture will be transmitted via the Waves
   low rate science flight software or high rate science flight software. 
   
LO_SEQ_NO:
   MSB first 16bit unsigned integer containing the least sequence number of
   the Waves IDP packets that contained data for this capture.
   
HI_SEQ_NO:
   MSB first 16bit unsigned integer containing the greatest sequence number of
   the Waves IDP packets that contained data for this capture.  For captures
   that were never segmented, this should be identical to LO_SEQ_NO.
   
LO_REPORT_SCLK:
   Numerically the smallest IDP header SCLK from the raw packets that make 
   up this data packet.

HI_REPORT_SCLK:
   Numerically the largest IDP header SCLK from the raw packets that make 
   up this data packet.

COLLECT_SCLK: 
   The SCLK at the time of the data collection, gathered from the
   IDP header SCLK and the mini-packet RTI field, takes into account the fact 
   that the last bit of the SCLK is a time quality field.

COL_RTI: 
   The Real Time Interrupt during which data collection was initiated for this
   capture.  There are 40 RTI's per second in the Waves instrument.  This is a
   number between 0 and 39.  Can be converted to factional seconds via:
   RTI/40.0.

AVG:
  The number of original data captures averaged to arrive at this capture.

IDP_CRC:
   An unsigned 16bit LSB first integer containing the CRC from the IDP header
   containing the initial packet in the dataset. 
   
   * WARNING:  Waves generated CRC's are little endian, GSE CRC's are big
               endian. *
   
MPH0-7:
  The minipacket header.  After unpacking, these are the original bits
  describing the data and extended headers.  As processing stages are 
  completed these may change.  These bits are broken out as follows:
  
            bit7    bit6    bit5    bit4    bit3    bit2    bit1    bit0
         +---------------------------------------------------------------+
   byte0 | ID3   | ID2   | ID1   | ID0   | Len11 | Len10 | Len9  | Len8  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte1 | Len7  | Len6  | Len5  | Len4  | Len3  | Len3  | Len1  | Len0  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte2 | RTI15 | RTI14 | RTI13 | RTI12 | RTI11 | RTI10 | RTI9  | RTI8  |
         +---------------------------------------------------------------+

         +---------------------------------------------------------------+
   byte3 | RTI7  | RTI6  | RTI5  | RTI4  | RTI3  | RTI3  | RTI1  | RTI0  |
         +---------------------------------------------------------------+
  
     GLOP=0
         +---------------------------------------------------------------+
   byte4 | GLOP  | PATN  | BGAIN | SEOF  |SEGNO3 |SEGNO2 |SEGNO1 |SEGNO0 |
         +---------------------------------------------------------------+
  
     GLOP=1
         +---------------------------------------------------------------+
   byte4 | GLOP  | PATN  | BGAIN |   1   | CYCL3 | CYCL2 | CYCL1 | CYCL0 |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte5 | ATTN3 | ATTN2 | ATTN1 | ATTN0 | SRC3  | SRC2  | SRC1  | SRC0  |
         +---------------------------------------------------------------+
  
     GLOP=0
         +---------------------------------------------------------------+
   byte6 |  BND7 | BND6  | BND5  | BND4  | BND3  | BND2  | BND1  | BND0  |
         +---------------------------------------------------------------+
  
     GLOP=1
         +---------------------------------------------------------------+
   byte6 | SEC7  | SEC6  | SEC5  | SEC4  | SEC3  | SEC2  | SEC1  | SEC0  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte7 | MSF1  | MSF0  | FMT5  | FMT4  | FMT3  | FMT2  | FMT1  | FMT0  |
         +---------------------------------------------------------------+
         
   ID
   --
   The packet ID, may be one of the following
    
      0x3 - HFWBR Baseband Waveforms from HFR process
      0x4 - HFR Spectra
      0x5 - HFR Baseband Spectra 
      0x7 - HFWBR Baseband Waveforms
      0x8 - HFWBR I or Q mixed Waveforms
      0x9 - LFR Spectra
      0xA - LFR Waveforms
   
   Len
   ---
   The data length.  Initially this in the amount of minipkt data - 1.
   This field is zero'ed by the unsegmentor.
 
   RTI
   ---
   This is a combination of the 40Hz interrput counter plus the SCLK as
   follows:
   
     RTI field = (0x3FF & SCLK)*40 + rti
    
   So to get the fraction of a second at which data collection was initiated:
    
      frac = (pMiniPkt->uRTI % 40)/40.0;
    
   This field can be used in conjunction with the IDP_SCLK to determine
   when data collection was initiated for this minipacket.
   
   GLOP
   ----
   If this bit is 1, glopping is enabled and the lower 4 bits of MPH4
   give the number of datasets that have been glopped together.  If this
   bit is 0, glopping is disabled and the lower 4 bits of MPH4 give the
   segment number.
   
   Note: The deglopper program sets this value to 0, zeros out any globbed
   attenuator settings bytes in MS_G, and adjusts the BND field appropriately.
   
   PATN
   ----
   If 1 the preamp attenuator is on.  The attenuation is 25dB for the Lo E
   and B LFR channels, and 20dB for LFR Hi E, HFR and HFWBR.
   
   BGAIN
   -----
   If 1 the back end LFR gain amp is on for the Hi and Lo E channels which
   adds 20dB to the signal.  This field should be ignored for HFR or HFWBR
   data.
   
   SEOF
   ----
   If 1 this is the last segment for a dataset.  After the unsegment step
   this bit is always 1.
   
   SEGNO (If GLOP = 0)
   -----
   The segment number for datasets that are too large to fix in a single
   minipkt.  The first segment is no. 0.
   
   CYCL (If GLOP = 1)
   ----
   The cycle count.  This is the number of datasets in the data section
   minus 1.  It is also the number of extra attenuator settings bytes in
   the MS_G field.
   
   ATTN
   ----
   The step number for the HFR or HFWBR attenuators.  The attenuation is 
   determined by:
   
       Attenuation = ATTN * 2 dB.
   
   This field should be ignored for LFR data.

   SRC
   ---
   The data source within the instrument, values are:
   
     0x1 - LFR B channel @ 50kHz
     0x2 - LFR Lo E channel @ 50kHz
     0x3 - LBR Hi E channel @ 375kHz
     0x4 - (unused)
     0x5 - (unused)
     0x6 - HFWBR Log Detector
     0x7 - HFR Log Detector
     0x8 - HFWBR I channel @ 1.3125MHz
     0x9 - HFWBR Q channel @ 1.3125MHz
     0xA - HFWBR Unknown channel @ 1.3125MHz
     0xB - HFWBR I channel @ 7MHz
     0xC - HFR I channel @ 1.3125MHz
     0xD - HFR Q channel @ 1.3125MHz
     0xE - HFR Unknown channel @ 1.3125MHz
     0xF - HBR I channel @ 7MHz
   
   BND (If GLOP = 0)
   ---
	This byte is sometimes the mixer seting, sometimes an indication of noise
	filtering.
	
	For HFR and HFWBR Reciever packets (PKTID = 0x3, 0x4, 0x5, 0x7, 0x8 )
	---------------------------------------------------------------------
   The mixer setting.  Values are:
   I 
     0x00 - HFR or HFWBR reciever is in baseband, A/D converter runs at 7MHz
     
     0x01 - HFR or HFWBR reciever is sweeping for log detection (Making HFR
            spectrum packets)
            
     0x0C to 0xBF - Mixer setting in MHz.  Note that the mixer runs at four
            times the center frequency for HFWBR I/Q packets thus:
            
            BND / 4.0 = Center MHz
     
   To get the band values when GLOP = 1, if the packet ID is 0x4 (HFR Specturm),
   treat as if BND were 0x01, if the packet ID is 0x5 (HFR Baseband Spectrum)
   treat as if BND were 0x00.  Waveforms are never glopped.
	
	For LFR Receiver packets (PKTID = 0x9, 0xA, some 0xF)
	-----------------------------------------------------
	   0xF0 - Suppressed Noise Filtering: pkt contains channel S+N only
		
		0xF1 - Preform Noise Filtering: pkt contains channel (S+N) - f(N) only
		
		0xF3 - Preform Noise filtering and deliver all diagnostic components:
		       pkt cantains three channels: S+N, N, S+N - f(N)
		
   
	
   SEC (If GLOP = 1)
   ---
   The time between cycles, expressed as seconds - 1 when GLOPping multiple
   datasets into a single minipacket.
   
   MSF
   ---
   The number of extra header bytes.  These contain extra info that is TBD.
   This is an enumeration as follows:
   
     MSF = 0, No MSF bytes, GLOP Attenuator bytes (if any) start at MS_G0.
     MSF = 1, 2 MSF bytes, GLOP Attenuator bytes (if any) start at MS_G2.
     MSF = 2, 4 MSF bytes, GLOP Attenuator bytes (if any) start at MS_G4.
     MSF = 3, 8 MSF bytes, GLOP Attenuator bytes (if any) start at MS_G8.
   
   FMT
   ---
   The current Data Section format.  FMT is a single byte with a maximum value
   of less than 128, so it may be read as either signed or unsigned.  See 5.3,
   Format Enumeration, for a list of format IDs.


MS_G0-22:
  Extra header bytes.  MSF bytes start first, and GLOP attenuator bytes follow.
  The maximum number of extra header bytes occurs if the MSF bit field is 3,
  and the GLOP bit is set and the CYCL bits = 0xF.  This gives 23 extra bytes.
  The minimum number of extra header bytes is 0.
  
  Glopped attenuator bytes have the format:
  
          bit7    bit6     bit5    bit4    bit3    bit2    bit1    bit0
     +---------------------------------------------------------------+
     | ATTN3 | ATTN2 | ATTN1 | ATTN0 |       |       | PATN  | BGAIN |
     +---------------------------------------------------------------+
     
  Where ATTN, PATN, and BGAIN are defined as above.
  
  Note: The deglopper zero's out any glopped attenuator bytes.


FLAGS:
  TBD


4.3.1.2 Memory Readout Block
----------------------------
This is only present for memory readout blocks.  It is not present if the
scince data status block is present.  The data body consists of a memory 
dump at the address given it this header.


Block Size:  56 bytes
Block Start: Offset 16 from the start of the GSE header.

       +--------+--------+--------+--------+--------+--------+--------+--------+
016-023|  0x11  |  0x1F  |MODIFIER| COL_RTI|           COLLECT_SCLK            |
       +--------+--------+--------+--------+--------+--------+--------+--------+
024-031|     IDP_CRC     |     SEQ_TS      |            REPORT_SCLK            |
       +--------+--------+--------+--------+--------+--------+--------+--------+
032-039|  MPH0  |  MPH1  |  MPH2  |  MPH3  |  MPH4  |  MPH5  |  MPH6  |  MPH7  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
040-047|  MSF0  |  MSF1  |  MSF2  |  MSF3  |  MSF4  |  MSF5  |  MSF6  |  MSF7  |
       +--------+--------+--------+--------+--------+--------+--------+--------+

MODIFIER:
   The ID of the last processing step to modify this block.

COLLECT_SCLK:
   The SCLK at the time of the memory dump, gathered from the IDP header SCLK
   and the mini-packet RTI field, takes into account the fact that the last bit
   of the SCLK is a time quality field.

IDP_CRC:
   The original CRC generated by the instrument for the IDP packet that 
   contained this memory read out.  This is close to useless.

SEQ_TS:
   This is a 16-bit MSB first unsigned integer. 

   * The upper 2 bits of this field contain the telemetry source.  This is
   retained so that we always know where the packets originated from.  This
   sub field may have the following values:
   
      0x1 - High Rate Science
      0x2 - Low Rate Science
      0x3 - Housekeeping
    
   For MRO data This should always have the value 0x2 for Low Rate Science.
   
   * The lower 14 bits of this field contain the sequence number.
   

REPORT_SCLK:
   MSB first 32-bit unsigned integer containing the spacecraft sclk at the
   time the IDP packet containing this MRO was reported out of the instrument.

MPH0-7:
  The minipacket header.  After unpacking, these are the original bits
  describing the data and extended headers.  These bits are broken out as
  follows:
  
            bit7    bit6    bit5    bit4    bit3    bit2    bit1    bit0
         +---------------------------------------------------------------+
   byte0 |   1   |   1   |   1   |   0   | Len11 | Len10 | Len9  | Len8  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte1 | Len7  | Len6  | Len5  | Len4  | Len3  | Len3  | Len1  | Len0  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte2 | RTI15 | RTI14 | RTI13 | RTI12 | RTI11 | RTI10 | RTI9  | RTI8  |
         +---------------------------------------------------------------+

         +---------------------------------------------------------------+
   byte3 | RTI7  | RTI6  | RTI5  | RTI4  | RTI3  | RTI3  | RTI1  | RTI0  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte4 |                               | FRAME3| FRAME2| FRAME1| FRAME0|
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte5 |  A19  |  A18  |  A17  |  A16  |  A15  |  A14  |  A13  |  A12  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte6 |  A11  |  A10  |   A9  |   A8  |   A7  |   A6  |   A5  |   A4  |
         +---------------------------------------------------------------+
  
         +---------------------------------------------------------------+
   byte7 | MSF1  | MSF0  | REG1  | REG0  |   A3  |   A2  |   A1  |   A0  |
         +---------------------------------------------------------------+
         

   Len
   ---
   The data length.  Initially this in the amount of minipkt data - 1.
   This field is zero'ed by the unsegmentor.
 
   RTI
   ---
   This is a combination of the 40Hz interrput counter plus the SCLK as
   follows:
   
     RTI field = (0x3FF & SCLK)*40 + rti
    
   So to get the fraction of a second at which MRO was read:
    
      frac = (pMiniPkt->uRTI % 40)/40.0;
    
   This field can be used in conjunction with the IDP_SCLK to determine
   when the memory read was initiated for this minipacket.

   FRAME
   -----
   If bit A19 is set to 1 then the address is within one of the 16 FFT
   512 MB memory frames.  This indicates which WvFE frame was currently
   mapped into the upper Y180 memory region.

   A19-A0
   ------
   The lowest memory address for the dump.  The first byte in the data 
   section was taken from this address.  Subsequent bytes are from 
   incrementing addresses.
  
   MSF1-0
   ------
   The number of extra header bytes.  These contain extra info that is TBD.
   This is an enumeration as follows:
   
      MSF = 0, No MSF bytes
      MSF = 1, 2 MSF bytes
      MSF = 2, 4 MSF bytes
      MSF = 3, 8 MSF bytes
     
   REG
   ---
   Most of the lower half of the Y180 memory space may be swapped between
   one of four memory regions.  Each of which is 512 MB in length.  This
   field indicates from which region the memory was dumped:
     
     REG = 0, Main Memory
     REG = 1, Program Memory
     REG = 2, IEB Bulk Memory
     REG = 3, Scratch Memory
         
   There is a suspicion that REG must = 0 if the address is less than 0x5000.

MSF0-7:
  Extra header bytes.  Though the "slot" for all 8 MSF bytes is always present
  the number that are valid is defined by the MSF subfield inside the MPH
  bytes.
  

4.3.1.3 Juno Flight Data Systems CCSDS header block
---------------------------------------------------
This block is attached to our data packets by the flight data systems.  We 
won't have access to it after ATLO.  For more details see section 3.4.2.3 of
the Juno "Flight-to-Ground Interface Control Document".  This is usually
present in housekeeping data, which doesn't have an SDS block created.  Offset
indicies are indicated assuming housekeeping data.

Block Size:   13 bytes
Block Start:  Offset 16 if the Science data status block is not present,
              Offset 72 if the Science data status block is present.
              I 
       +--------+--------+--------+--------+--------+--------+--------+--------+
016-023|  0x20  |  0x0C  |    FLAGS_APID   |  FLAGS_PART_NO  |  CCSDS_PKT_LEN  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
024-031|               READ_SCLK           |READ_FRC|
       +--------+--------+--------+--------+--------+


FLAGS_APID: 
   This is a 16-bit unsigned MSB first field with the following sub-fields:
   
   |          BYTE OFFSET 2        |        BYTE OFFSET 3          |
   | 15  14  13  12  11  10   9   8|  7   6   5   4   3   2   1   0|
   +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
   |    VER    |TYP|SHF|                  APID                     |
   +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
            
   VER - The CCSDS version number, always 0.
  
   TYP - Always 0, indicating telemetry data.
  
   SHF - Secondary header flag, always 1.
  
   APID - May have one of the following values (decimal):
      96 - Waves Housekeeping data
      97 - Waves High Rate Science
      98 - Waves Low Rate Science
      99 - Waves Unknown


FLAGS_PART_NO:
   This is a 16-bit unsigned MSB first field with the following sub-fields:
   
   |          BYTE OFFSET 4        |        BYTE OFFSET 5          |
   | 15  14  13  12  11  10   9   8|  7   6   5   4   3   2   1   0|
   +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
   |GRPFLAG|                   PRODUCT_PART_NO                     |
   +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

   
   GRP_FLAG - Always 3, non-segmented data.
   
   PRODUCT_PART_NO - 1 for the first waves packet in a "product", grows 
      from there, up to the maximum number of packets in a "product".


CCSDS_PKT_LEN:
   A 16-bit MSB first unsigned integer indicating the size of the contained
   waves packet + 4.


READ_SCLK:
   A 32-bit MSB first unsigned integer containing the SCLK value at which the
   Waves packet was read by the Juno FDS.  The Juno SCLK is defined as seconds
   since 2001-01-01T12:00:00. (i.e. Noon of the 1st day of last year of the
   century.)
   
   
READ_FRC:
   The fractional seconds of the read time * 256.  To get the fractional
   second divide this value by 256.


4.3.1.4 Juno Flight Data Systems High Rate Bin Header
-----------------------------------------------------
Each high rate data recording session has a bin header prefix attached on the
spacecraft.  During ground processing, this bin header is copied into each
packet to which it applies as a header block.  That way if a packet were to be
separated from the recording session file, the information is preserved.  This
header block is usually the first block for data in *.hrs.raw files, it is
usually the second item in *.hrs.ucal files.  Since the block positition is
not fixed, offsets are given from the beginning of the block instead of the
start of the packet.  Note that blocks are *always* serialized into the header
in asscending order by thier block type ID.

Block Size:   24 bytes
Block Start:  Varies depending on presence of other blocks

       +--------+--------+--------+--------+--------+--------+--------+--------+
000-007|  0x1A  |  0x17  |     PKT_IDX     |            Q_FACTOR_SEC           |
       +--------+--------+--------+--------+--------+--------+--------+--------+
008-015| Q_FACTOR_SUBSEC | JUNO_PROC_SUBSEC|          JUNO_PROC_SEC            |
       +--------+--------+--------+--------+--------+--------+--------+--------+
016-023|           PRE_EVENT_WORDS         |Q_FACTOR|  0x00  |  0x00  |  0x00  |
       +--------+--------+--------+--------+--------+--------+--------+--------+

PKT_IDX:
   An unsigned 16bit MSB first integer containing the index of this packet in
	the bin file.  The first packet in the bin file is index number 0.

Q_FACTOR_SEC:
	An unsigned 32bit MSB first integer containing the seconds field of the
	timestamp sent by the Waves instrument with the quality factor that caused
	the bin to be saved.  For bins filled with initialization data for which 
	there is no Waves quality factor, this value is 0.

Q_FACTOR_SUBSEC:
   An unsigned 16bit MSB first integer containing the subseconds field of the
	timestamp sent by the Waves instrument with the quality factor that caused
	the bin to be saved.  Each count in this field represents 1/(2^16) seconds.
	For bins filled with initialization data for which there is no Waves quality
	factor, this value is 0.
	
JUNO_PROC_SUBSEC:
	An unsigned 16bit MSB first integer containing the subseconds field of the
	time when either 1) the spacecraft software processed the Waves quality
	factor that caused the bin, or 2) for bins with data with no quality factor,
	the time when the bin completed filling.

JUNO_PROC_SEC:
	An unsigned 32bit MSB first integer containing the seconds field of the time
	when either 1) the spacecraft software processed the Waves quality factor
	that caused the bin, or 2) for bins with data with no quality factor, the
	time when the bin completed filling.

PRE_EVENT_WORDS:
	An unsigned 32bit MSB first integer containing the number of words in the
	saved bin preceding the quality point.  Nominally will be the configured
	value, but may be fewer if there are fewer words in the bin when the Waves
	quality factor is received.  Will be 0 if the Waves quality factor is 0,
	indicating no quality factor.

Q_FACTOR:
	An unsigned byte containing the Quality factor associated with the saved bin
	as received from Waves, or 0 if there is no quality factor.



4.3.2 Generic Reader Processing Block  
-------------------------------------
This block records when data was originally read from the up stream source.
For unsegmented captures, this is the reader block that corresponds to the
first segment the capture.  See 4.3.3.4,  Unsegmenter Processing Block to find
original read times for unsegmented packets.

Block Size:   13 bytes
Block Start:  Offset 72 from the start of the GSE header unless CCSDS header
              was copied in.

       +--------+--------+--------+--------+--------+--------+--------+--------+
072-079|  0x70  |  0x0F  |   0x10 | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
080-087|BEG_TM_US_DIV_100| ERRORS |WARNINGS| SOURCE |
       +--------+--------+--------+--------+--------+
       
See 4.3.2.2 for common processing block field definitions. 

SOURCE:
  The lower 4 bits contain the source of the data.  This has the following
  enumeration that will be expanded as necessary.
  
    0x1  Lockheed Martin Juno SCE, Low Rate Interface
    0x2  Lockheed Martin Juno SCE, High Rate Interface
    0x3  A File
	 0x4  direct SOPC IP query
	 0x5  direct TIS IP connection
      
      
4.3.2.1 Lockheed Martin SCE LRS Interface Reader Processing Block
-----------------------------------------------------------------
This block records which SCE as well as the IP/UDP/CIP information recieved
from the SCE.  Offsets are given for unpacked data.  For Raw Scicence and Raw 
housekeeping, subtract 64.

Block Size:  60 bytes
Block Start: Offset 72 from the start of the GSE header

       +--------+--------+--------+--------+--------+--------+--------+--------+
072-079|  0x70  |  0x3B  |  0x11  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
080-087|BEG_TM_US_DIV_100| ERRORS |WARNINGS| SOURCE |        PROCESS_UID       |
       +--------+--------+--------+--------+--------+--------+--------+--------+
088-095|          SOCKET_REMOTE_IP         |          SOCKET_LOCAL_IP          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
096-103|                                                                       |
       +                         IP Header                                     +
104-111|                                                                       |
       +                                   +--------+--------+--------+--------+
112-119|                                   |        UDP Header first 4 bytes   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
118-127|     UDP Header last 4 bytes       |        CIP Header first 4 bytes   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
128-135|     CIP Header last 4 bytes       |
       +--------+--------+--------+--------+

PROCESS_UID:
  The 32 bit user ID of the workstation account under which this program was
  run.

SOCKET_REMOTE_IP:
  A 32 bit unsigned MSB first integer containing the IPv4 address of the remote
  end (SCE end) of the TCP/IP socket from which the data was collected.

SOCKET_LOCAL_IP:
  A 32 bit unsigned MSB first integer containing the IPv4 address of the local
  end (workstation end) of the TCP/IP socket from which the data was collected.
  
IP Header:
  The 20 byte IP header generated by waves and sent to the SCE Low Rate Science
  interface.
  
UDP Header:
  The 8 byte UDP header generated by Waves and sent to the SCE Low Rate Science
  interface.
  
CIP Header:
  The 8 byte CIP header generated by Waves and sent to the SCE Low Rate Science
  interface.
 

4.3.2.2 Lockheed Martin SCE HRS Interface Reader Processing Block
-----------------------------------------------------------------
This block records which SCE as well as the HRS sync pattern.  

Block Size:   28
Block Start:  Offset 72 from the start of the GSE header

       +--------+--------+--------+--------+--------+--------+--------+--------+
072-079|  0x70  |  0x1B  |  0x12  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
080-087|BEG_TM_US_DIV_100| ERRORS |WARNINGS| SOURCE |        PROCESS_UID       |
       +--------+--------+--------+--------+--------+--------+--------+--------+
088-095|          SOCKET_REMOTE_IP         |          SOCKET_LOCAL_IP          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
096-103|  0xFA  |  0xF3  |  0x34  |  0x03  |
       +--------+--------+--------+--------+
       
See the description of the LRS Interface Reader Processing Block for field 
definitions


4.3.2.3 JPL Product file Reader Processing Block
------------------------------------------------
This block records the input filename from which the packet was derived, as 
well as the generic items.

Block Size:  60 bytes
Block Start: Offset 72 from the start of the GSE header

       +--------+--------+--------+--------+--------+--------+--------+--------+
072-079|  0x70  |  0x3B  |  0x13  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
080-087|BEG_TM_US_DIV_100| ERRORS |WARNINGS|  0x03  |  TRUNC |                 |
       +--------+--------+--------+--------+--------+--------+                 +
088-095|                                                                       |
       +                                                                       +
096-103|                                                                       |
       +                                                                       +
104-111|                  FILENAME (up to 45 characters)                       |
       +                                                                       +
112-119|                                                                       |
       +                                                                       +
118-127|                                                                       |
       +                          +--------+--------+--------+--------+--------+
128-135|                          |  0x00  |
       +--------+--------+--------+--------+

See the generic reader block fields at 4.3.2 for a description of the PROGRAM,
BEGIN_TIME_SEC, BEG_TM_US_DIV_100, ERRORS, and WARNINGS fields.

The SOURCE of these packets is a file, so byte offset 12 from the start of the
block is always 0x03.

FLAGS:
   The upper four bits are 0x1, if the filename was truncated, 0x0 otherwise.
   The lower four bits are 0x2, if the filename field contains 2-byte wide 
   characters.  The lower four bits are 0x1 for one byte wide characters.

FILENAME:
   This is the ascii (or unicode) base name of the file.  I.E. no directory
   components.  The name is left-justified in the field.  NULL is used for the
   padding character.
   
   The filename is *ALWAYS* null terminated.  A trailing null is included in
   the block definition to insure this.


4.3.2.4 JPL SFDU Reader Processing Block
--------------------------------------------
This block is generated by readers than take in SFDU packets, it may hold 
either IP address information or a filename, some selected SFDU fields are
broken out.

Block Size:  112 bytes

Information used to generate this block was taken from:

  "SAS to IOT, Packetized Instrument Engineering Data File", SIS-ID: SCI018

       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  |  0x6f  |   0x14 | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r15|BEG_TM_US_DIV_100| ERRORS |WARNINGS| SOURCE |   VER  |  TRUNC |        |
       +--------+--------+--------+--------+--------+--------+--------+        +
r16-r23|                                                                       |
       +                                                                       +
r24-r31|                                                                       |
       +                                                                       +
r32-r39|                                                                       |
       +                                                                       +
r40-r47|                         SOURCE_NAME                                   |
       +                                                                       +
r48-r55|                                                                       |
       +                                                                       +
r56-r63|                                                                       |
       +                                                                       +
r64-r71|                                                                       |
       +                                                              +--------+
r72-r79|                                                              |  0x00  |
		 +--------+--------+--------+--------+--------+--------+--------+--------+
r80-r87| 				 OFFSET  				 |  FLAGS | 	 SCFT_ID 	 | DSN_SRC|
       +--------+--------+--------+--------+--------+--------+--------+--------+
r88-r95| 						EARTH_RECEIVE_TIME					 |   BIT_RATE...   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r96-103| 	 ...cont 	 | 						TIS_RECORD_TIME						 |
       +--------+--------+--------+--------+--------+--------+--------+--------+
104-111|                        TIS_PUB                      |  NON_FILL_LEN   |
       +--------+--------+--------+--------+--------+--------+--------+--------+


SOURCE:
	The NULL terminated ASCII data source, one of: 
	   3 - data is from a file, and SOURCE_NAME is the filename.
	   4 - remote SOPC query IP , TIS connection IP, and SOURCE_NAME is the
		    ASCII remote hostname or IP and port number.

VER:
  The version of the SFDU reader header, 0x01 for now.
  
TRUNC:
  If 1, the SOURCE_NAME was truncated to 64 bytes, if 0 the SOURCE_NAME is not
  truncated.

SOURCE_NAME:
   ASCII NULL Terminated text containing the filename or remote IP of the
	source of this packet.
	
OFFSET:
	The 0 based, byte offset into the source at which this SFDU packet was read.

FLAGS:
   A bit field:
	
	  bit 7    bit 6    bit 5    bit 4    bit 3    bit 2    bit 1    bit 0  
	+--------+--------+--------+--------+--------+--------+--------+--------+
	|                                            | RETRANS|   FILL |   PAD  |
	+--------+--------+--------+--------+--------+--------+--------+--------+

		RETRANS
		-------
		Taken from byte 10, bit 1 (IBM counting) of the Secondary CHADO
		(if present).  When set to 1 the frame containing the start of the 
		packet had the retransmission flag set, otherwise 0
		
		FILL
		----
		Taken from byte 4, bit 0 (IBM counting) of the Tertiary CHADO
		(if present).  When set to 0 the data section consists entirely
		of data (with the possible exception of the pad byte), when set to
		1 the packet contains fill.  This should ALWAYS be zero as Juno
		is not planning to utilize this capability for routine operations.
		
		PAD
		---
		Taken from byte 4, bit 1 (IBM counting) of the Tertiary CHADO (if
		present).  Since SFDUs must be an even number of bytes, this bit
		indicates if the original SFDU's Data CHADO had a pad byte at the
		end.  If 0 it was not padded, if 1 the data chado was padded.
		

SCFT_ID:
   An MSB 1st 2 byte unsigned integer containing the spacecraft ID.   Has
	one of two values:
	
	     61 - really from Juno
		  63 - from the STL simulator

DSN_SRC:
	An unsigned byte indicating the number of the DSN tracking station which
	received this data.   Here's a breakdown by value:
	
	Goldstone
	DSS-24  34M Beam Wave Guide (BWG)
	DSS-25  34M BWG
	DSS-26  34M BWG
	DSS-15  34M High Efficiency (HEF)
	DSS-14  70M

	Canberra
	DSS-34  34M BWG
	DSS-45  34M HEF
	DSS-43  70M

	Madrid
	DSS-54  34M BWG
	DSS-55  34M BWG
	DSS-65  34M HEF
	DSS-63  70M

   From an Aug. 2011 Email from Amy Hale
	
		 
EARTH_RECEIVE_TIME:
	The 6-byte Earth Receive Time, see 820-013, 173-Telecomm-Time to decode
	these bytes.

BIT_RATE:
	An IEEE MSB 1st 4 byte float containing the actual data bit rate as
	measured by the DIT (which is???).
	
TIS_RECORD_TIME:
	The system clock time at which the TIS system generated this packet.
	See 820-013, 173-Telecomm-Time to decode these bytes.
	
TIS_PUB:
	An ASCII string (optionally) entered into the TIS interface by user
	control.  This field is handy for recording test numbers in ATLO.
	
NON_FILL_LEN:
	2-byte MSB 1st unsigned integer indicating the number of bytes in the
	data chado (after the type and length fields) of valid data.
	

4.3.3 Waves Science IDP Unpacker Processing Block  
-------------------------------------------------
This is a standard processing block which states when and which program handled 
the unpacking process.  Captures that have been unsegmented no longer have this
block.  See 4.3.3.4, Unsegmenter Processing Block to find original unpacker 
program IDs.

Block Size: 13 bytes
Block Start: End of previous block + 1

       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  |  0x0F  |  0x20  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r12|BEG_TM_US_DIV_100| ERRORS |WARNINGS| NUMPKTS| PKT_IDX|                 |
       +--------+--------+--------+--------+--------+--------+--------+--------+

NUMPKTS:
  The number of minipackets in the original IDP.
  
PKT_IDX:
  The index of the minipacket in the original waves packet from which it was
  derived.  NOTE:  Versions of JWavesUpkFilter Prior to 2011-08-12 DID NOT FILL
  OUT THIS FIELD.  Normally it doesn't matter, but some MRO packets need this
  field in order to distinguish consecutive dumps of the same memory area within
  a single RTI.


4.3.4 Unsegmenter Processing Block
----------------------------------
The program which handles the unsegmentation process fills in this block. 
Certain parameters from the Reader and Unpacker blocks are contained here as 
those blocks are not unique for unsegmented captures and are thus not present in
the header for unsegmented packets.

Block Size:  64 bytes
Block Start: End of previous block + 1

       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  |  0x3F  |  0x30  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r15|BEG_TM_US_DIV_100| ERRORS |WARNINGS|  FLAGS |LO_UP_ID|HI_UP_ID|        |
       +--------+--------+--------+--------+--------+--------+-----------------+
r16-r23|LO_RP_ID| LO_SRC |       LO_READ_BEGIN_TIME_SEC      |  LO_READ_TM_US  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r24-r31|HI_RP_ID| HI_SRC |       HI_READ_BEGIN_TIME_SEC      |  HI_READ_TM_US  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r32-r39|  Seq & TS No. 1 |  Seq & TS No. 2 |  Seq & TS No. 3 |  Seq & TS No. 4 |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r40-r47|  Seq & TS No. 5 |  Seq & TS No. 6 |  Seq & TS No. 7 |  Seq & TS No. 8 |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r48-r55|  Seq & TS No. 9 | Seq & TS No. 10 | Seq & TS No. 11 | Seq & TS No. 12 |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r56-r63| Seq & TS No. 13 | Seq & TS No. 14 | Seq & TS No. 15 | Seq & TS No. 16 |
       +--------+--------+--------+--------+--------+--------+--------+--------+

FLAGS:
     A bit field of flags, as follows:
     
          7       6       5       4       3       2       1       0
     +-------+-------+-------+-------+-------+-------+-------+-------+
     | NoSeg3| NoSeg2| NoSeg1| NoSeg0|       |       |       | NoSeg4|
     +-------+-------+-------+-------+-------+-------+-------+-------+
        
     NoSeg:  The number of segments combined to generate the complete capture.
     
     Note: this odd pattern for storing the bits it to make the field 
     backwards compatible, without re-producing .ucal data.
  
LO_UP_ID:
   The lowest program ID that preformed unpacking on data contained in this
   packet.  An unsigned byte.

HI_UP_ID:
   The greatest program ID that preformed unpacking on data contained in this
   packet.  An unsigned byte.
   
LO_RP_ID:
   The lowest program ID that read data contained in this packet from its 
   original source.  An unsigned byte.

LO_SRC:
   The lowest source ID for data contained in this packet.  An unsigned byte.

LO_READ_BEGIN_TIME_SEC:
   The earliest time of reading for data contained in this packet.  This is a 32
   bit unsigned MSB first integer.  It is the number of seconds elapsed since 
   00:00:00 on January 1, 1970, Coordinated Universal Time (UTC).
   
LO_READ_TM_US:
   The earliest time of reading for data contained in this packet. This is an   
   unsigned MSB first 16-bit twos compliment integer containing 1/100th the
   number of microseconds since the LO_READ_BEGIN_TIME_SEC.  To get the earliest
   read time microseconds, multiply this value by 100.

HI_RP_ID:
   The greatest program ID that read data contained in this packet from its 
   original source.  An unsigned byte.

HI_SRC:
   The greatest source ID for data contained in this packet.  An unsigned byte.

HI_READ_BEGIN_TIME_SEC:
   The latest time of reading for data contained in this packet.  This is a 32
   bit unsigned MSB first integer.  It is the number of seconds elapsed since 
   00:00:00 on January 1, 1970, Coordinated Universal Time (UTC).

HI_READ_TM_US:
   The latest time of reading for data contained in this packet. This is an   
   unsigned MSB first 16-bit twos compliment integer containing 1/100th the
   number of microseconds since the LO_READ_BEGIN_TIME_SEC.  To get the latest
   read time microseconds, multiply this value by 100.

Seq & TS No. 1 through 16:
    Contains the original sequence and telemetry source numbers of the IDP
    packets used to generate this unsegmented data.  These 16 bits contain
    two fields:
    
      15  14  13  12  11  10   9   8   7   6   5   4   3   2   1   0
    +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
    |   OS  |       Big Endian Sequence Number 0 to  16383          |
    +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
    
    The values for TS (Telemetry Source) are:
    
       0 - Invalid
       1 - High Rate Science
       2 - Low Rate Science
       3 - Housekeeping
    
    Up to 16 mini-packets can be combined into a single unsegmented mini-packet.
    If mini-packet data  can be up to 4 KB then this provides for 64 KB data
    sections.  Note that 0 is a legal value for at most one of the sequence
    numbers.  To distinguish between trailing zeros and a real sequence number
    consult the NoSeg bits of the FLAGS byte.
    


4.3.5 Post-Unsegmented Decompression Block
------------------------------------------I 
The decompression block is a simple extension of the Generic Processing Block
described in 4.2.4.  In addition to the standard processing block information,
the ratio of un-compressed to compressed data is stored.

Block Size:  16 bytes
Block Start: End of previous block + 1

       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  |  0x0F  |  0x40  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r16|BEG_TM_US_DIV_100| ERRORS |WARNINGS|          COMPRESSION_RATIO        |
       +--------+--------+--------+--------+--------+--------+-----------------+

COMPRESSION_RATIO:
   An MSB first IEEE-745 single precision floating point value contianing the
   number of bytes in the un-compressed data divided by the number of bytes in
   the compressed data.
   

4.3.6 Deglopper Block
---------------------
       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  |  0x0F  |  0x40  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r16|BEG_TM_US_DIV_100| ERRORS |WARNINGS|             (unused)              |
       +--------+--------+--------+--------+--------+--------+-----------------+


4.3.7 Sideband Recovery Block
-----------------------------
The I and Q channels must be combinded into a single packet to produce a 
viewable spectrum.  For these packets, there will usually be one or two reader
blocks, one unpacker blocks, no unsegment blocks and one decompression blocks.
The information in the unsegment blocks is subsumed into the sideband recovery
block.

Block Size: 94 bytes
Block Start: End of previous block + 1

       +--------+--------+--------+--------+--------+--------+--------+--------+
r00-r07|  0x70  |  0x5D  |  0x70  | PROGRAM|           BEGIN_TIME_SEC          |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r08-r15|BEG_TM_US_DIV_100| ERRORS |WARNINGS|PR0C_ID1|PROC_ID2|PROC_ID3|PROC_ID4|
       +--------+--------+--------+--------+--------+--------+-----------------+
r16-r23|       LO_READ_BEGIN_TIME_SEC      |       HI_READ_BEGIN_TIME_SEC      |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r24-r31|  LO_READ_TM_US  |  HI_READ_TM_US  | FLAGS  | NO_PKTS|    I SeqNo. 1   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r32-r39|    I SeqNo. 2   |    I SeqNo. 3   |    I SeqNo. 4   |    I SeqNo. 5   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r40-r47|    I SeqNo. 6   |    I SeqNo. 7   |    I SeqNo. 8   |    I SeqNo. 9   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r48-r55|    I SeqNo. 10  |    I SeqNo. 11  |    I SeqNo. 12  |    I SeqNo. 13  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r56-r63|    I SeqNo. 14  |    I SeqNo. 15  |    I SeqNo. 16  |    Q SeqNo. 1   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r64-r71|    Q SeqNo. 2   |    Q SeqNo. 3   |    Q SeqNo. 4   |    Q SeqNo. 5   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r72-r79|    Q SeqNo. 6   |    Q SeqNo. 7   |    Q SeqNo. 8   |    Q SeqNo. 9   |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r80-r87|    Q SeqNo. 10  |    Q SeqNo. 11  |    Q SeqNo. 12  |    Q SeqNo. 13  |
       +--------+--------+--------+--------+--------+--------+--------+--------+
r88-r95|    Q SeqNo. 14  |    Q SeqNo. 15  |    Q SeqNo. 16  |
       +--------+--------+--------+--------+--------+--------+

NO_PKTS:
   The upper bits are the number of I segments, the lower bits are the number
   of Q segments.


4.3.8 Calibration Block
-----------------------
TBD


4.4 Instrument Data
---------------------
Packet data values follow the end of the Ground Support header.  See section
5, GROUND SUPPORT DATA FORMAT for more information.


4.5 Total Packet Length Repetition
-----------------------------------
A repeat of the total packet length to enable backwards reading of ground
support files.
                 0       1       2       3    
           +-------+-------+-------+-------+ 
N-4 to N-1 |   Total Packet Length Repeat  | 
           +-------+-------+-------+-------+

   


5 GROUND SUPPORT DATA FORMAT
============================
The FORMAT field of the Science Data Status Block defines the storage type for
samples in the data section of a Waves ground packet.  It is the responsibility
of any program altering the data section to update this field. 


5.1 Byte Order
--------------
All data is stored in LITTLE ENDIAN (LSB first) order.


5.2 Format Parameters
---------------------
There are 5 semi-independently varying parameters that affect the data format.
  
   1. Has the data been Rice compressed or not, only samples 
      of 16 bits or less can be compressed. 
  
   2. What is the sample type, possible choices are:
      8-bit, 12-bit and 16-bit integers, and 9-bit pseudo-floats, 
      and 32-bit IEEE floats.  The 9-bit pseudo-float format
      is described below.
 
   3. What is the sample block size, possible choices are:
      8 bit integers, 9bits, 12bits, 16bits or 32bits.  There is

      one sample per sample block.  For Rice compressed data
      the block size does not apply.  The decompression 
      software is free to put the resulting samples into
      whichever block size it desires.
  
   4. What is the position of the sample within the sample 
      block? Is it shifted to high bits or to the low bits.
 
   5. For integer data, should it be considered signed or
      unsigned.
   
Since this 5 dimensional parameter space represents far more data formats than
will actually be used, the FMT field is an enumeration, with each value
representing a unique set of the format parameters.  See the table below for
enumerated values.
   
     
5.3 Format Enumeration
----------------------
          
     FMT      Sample   Rice   Block   Sampl   
     Val       Type    Comp   Size     Pos   Signed  Format Name
    -----   ---------  ----  -------  -----  ------  ----------------------  
     0x14    8bit int    on     -       -    Unsign  8bit_rice
      0x4    8bit int   off   8-bits    -    Unsign  8bit_in_8bit
     0x11    8bit int   off  16-bits  Lower  Unsign  8bit_in_16bit_lo
      
     0x13   12bit int    on     -       -    Unsign  12bit_rice
     0x10   12bit int   off   8-bits    -    Unsign  12bit_in_8bit_truc
      0x6   12bit int   off  12-bits    -    Unsign  12bit_in_12bit
      0x7   12bit int   off  16-bits  Lower  Unsign  12bit_in_16bit_lo
      0x8   12bit int   off  32-bits  Lower  Unsign  12bit_in_32bit_lo

      0x9   16bit int    on     -       -    Unsign  16bit_rice
      0xA   16bit int   off   8-bits    -    Unsign  16bit_in_8_bit_truc
      0xB   16bit int   off  12-bits    -    Unsign  16bit_in_12_bit_truc
 0x1C,0xC   16bit int   off  16-bits    -    Unsign  16bit_in_16_bit
      0xD   16bit int   off  32-bits  Lower  Unsign  16bit_in_32_bit_lo

      0xE*   9bit flt    on  12-bits  Lower  Unsign  9bit_in_12_bit_lo_rice
      0xF*   9bit flt    on  16-bits  Lower  Unsign  9bit_in_16_bit_lo_rice
      0x5    9bit flt   off   9-bits    -    Unsign  9bit_in_9bit
      0x3    9bit flt   off  16-bits  Lower  Unsign  9bit_in_16bit_lo
     0x12    9bit flt   off  32-bits  Lower  Unsign  9bit_in_32bit_lo
     
      0x2   32bit flt   off  32-bits    -    Signed  32bit_in_32bit
   
     * Formats 0xE and 0xF represent putting 9-bit pseudo-floats into
       a 12 or 16 bit block and compressing the blocks, we are unlikely
       to do this, but it is possible.     
    
    
5.4 9-bit Pseudo Float Format
-----------------------------
This value format has been designed to represent a relatively large range
of magnitudes in relatively few bits.  See section 4.3.1.6 of the "Juno
Waves Software Preliminary Detailed Design" document for complete information.

These 9 bits are stored in 16 bits as:
    
      MSB       LSB    
   +--------+--------+
   |       e|eeeeefff|
   +--------+--------+

Where the sign is always positive (and thus not included) and the exponent
excess is 20 instead of the standard IEEE-745 single precision value of 127.
For reference the IEEE-745 single precision encoding is:

      MSB                        LSB
   +--------+--------+--------+--------+
   |SEEEEEEE|EFFFFFFF|FFFFFFFF|FFFFFFFF|
   +--------+--------+--------+--------+

To convert from 9-bit pseudo floats to IEEE-745 encoding (often called 
float in C):

  1. Set the IEEE-745 sign bit (S) to 0 to represent a positive number
  
  2. Subtract 20 and add 127 to the pseudo float exponent (eeeeee) to
     get the IEEE-745 exponent (EEEEEEEE).
  
  3. Set the upper 3 bits of the IEEE-745 fraction to match the 
     the bits of the pseudo-float fraction (fff).
     
  4. Set the lower 20 bits of the IEEE-745 fraction to zero.



APPENDIX A: WAVES IDP
=====================
(To Be Included, see SVN_SERVER:/juno_hw/DPU_FSW/trunk/kernel/minipkt.inc 
 for now.)



APPENDIX B: CRC-CCITT
=====================
A C-code listing of the CRC-CCITT algorithm with constants used on Juno Waves

static int              crc_tabccitt_init = 0;
static unsigned short   crc_tabccitt[256];
#define                 P_CCITT     0x1021

static void init_crcccitt_tab( void ) {
   int i, j;
   unsigned short crc, c;

   for(i=0; i<256; i++) {
      crc = 0;
      c   = ((unsigned short) i) << 8;

      for(j=0; j<8; j++) {

         if( (crc ^ c) & 0x8000 ) 
            crc = ( crc << 1 ) ^ P_CCITT;
         else
            crc =   crc << 1;

          c = c << 1;
      }
      crc_tabccitt[i] = crc;
   }
   crc_tabccitt_init = 1;
}  

/* Calc CRC-CCITT on arbitrary len set of char data */
unsigned short crc_ccitt(const char* sData, size_t szLen){
   int i;
   unsigned short usIdx, usChar, usCRC;

   sRet[2] = '\0';
   
   if( ! crc_tabccitt_init ) init_crcccitt_tab();
    
   usCRC = 0;
   for(i = 0; i<szLen; i++){
      usChar  = 0x00ff & (unsigned short) sData[i];
      
      usIdx = (usCRC >> 8) ^ usChar;
      usCRC = (usCRC << 8) ^ crc_tabccitt[usIdx];
    }
      
   return usCRC;
}


APPENDIX C: PROGRAM IDS
=======================
Each program which handles Waves Data should record a Program ID in the 
appropriate spot in the GSE header.  Known program ID follow.


0x12  GSE/trunk/SCE/waves_sce_rdr.py
----  --------------------------------
This refers to Version two of the the sce reader script.  It gets data from
the Juno SCE's provided by Lockheed Martin corporation for the Juno instrument
teams.  


0x22  GSE/trunk/ground_proc/waves_unseg.py
----  ------------------------------------
Unsegmenter. (Not yet written)


0x31  GSE/trunk/ground_proc/waves_decomp.py
----  ------------------------------------
Post-Unsegmented, decompresser (Not yet written)


