API Documentation

States in NxCore

NxCore uses a concept of a State to allow the persisting of snapshot of the memory, and a subsequent loading of the memory to return to the exact spot in the processing.

A very key point to understand is that the "state" is just a starting point

If the state is from a real-time processing, and it's still the same day It will pick up from the timestamp of the state, but will switch to real-time when it catches up
If the state is from a real-time processing, and it's a different day It will pick up from the timestamp of the state, and continue with the tape
If the state is from a tape It will pick up from the timestamp of the state, and continue with the tape

Notes

  • While taking a state snapshot is fast, it's not free, and you have to balance your performance requirements against your need to be able to recover from a crash without having the replay the entire tape.
  • The memory saved includes the UserData1 and UserData2 fields of every NxString.
    • If these fields contain primitive values, they save and load without any issues. This includes pointer values, but not the pointed-to-areas
    • If these fields contain pointers, the saving will NOT also save the pointed-to areas, as NxCore has no idea of their size and contents. The example below gives one solution to persisting your own data successfully.
  • Easy way to tell if you are running a state or realtime or tape, is if the very first callback you get is:
    • pNxCoreSys->Status==NxCORESTATUS_LOADED_STATE, you're processing a state
    • pNxCoreSys->Status==NxCORESTATUS_INITIALIZING, you're running real-time or processing a tape

Saving States

You can save a state anytime within your callback by calling the nxCoreClass.SaveState() method and passing in the file name to save into and a flag telling it how you want the data for saving collected and written out.

#define Value Comments
NxSAVESTATE_GRADUALLY 0 Save completes in multiple passes - NxCore writes out memory block to file in 1MB increments.
NxSAVESTATE_ONEPASS 1 Save completes in one pass - NxCore writes out entire memory block to file at one time.
NxSAVESTATE_CANCEL 2 Abort a scheduled Save State -- can be done at NxCSSAVESTATE_CAPTURE time

For example, nxCoreClass.SaveState(stateFileName, NxSAVESTATE_ONEPASS) triggers a number of subsequent calls into your callback with pNxCoreSys->Status==NxCORESTATUS_SAVING_STATE with pNxCoreSys->StatusData one of:

If Status: StatusData:
NxCORESTATUS_SAVING_STATE
#define Value Comments
NxCSSAVESTATE_CAPTURE 0 the core state will be captured after you return from this callback
NxCSSAVESTATE_COMPLETE 1 the save core state operation is complete
NxCSSAVESTATE_ERR_CREATEFILE -1 failed to open the specified tape filename
NxCSSAVESTATE_ERR_DISKSPACE -2 not enough disk space to complete the operation
NxCSSAVESTATE_ERR_MEMORY -3 insufficient memory to complete the operation

Loading States

To load a state, you'd just pass the file name in to the nxCoreClass.ProcessTape, just as you would pass in "" for real-time, or a tape.

The example below shows how to pre-allocate a specific, unused, memory address to ensure that the state can load in the same memory it was saved in.

Unlike processing a real-time feed or a tape, the first call to your callback function will be with pNxCoreSys->Status==NxCORESTATUS_LOADED_STATE, and pNxCoreSys->StatusData will contain the size of the tape inside the file. The example below shows how to use that "size" to persist and retrieve additional data to the end of the file.

Example

This example assigns a unique value to each symbol in Userdata1 on the initial symbol spin and processes trades until 9:00 AM EST, when it will save the current state to a state tape file and exit the ProcessTape thread. Next it will start the ProcessTape thread again, this time starting it with the newly saved state tape and continue to process trades. Note that the UserData1 values have persisted in the saved state tape.

include "NxCoreLoadLib.h"  // NxCore DLL Load Library    

// Couple forwards - functions are below
void OnNxCoreStatus(const NxCoreSystem* pNxCoreSys,const NxCoreMessage* pNxCoreMsg);
int __stdcall OnNxCoreCallback(const NxCoreSystem* pNxCoreSys,const NxCoreMessage* pNxCoreMsg);
void OnNxCoreTrade(const NxCoreSystem* pNxCoreSys, const NxCoreMessage* pNxCoreMsg);    

// Couple globals    
unsigned char SavedState=0;
char StateFileFilename[255];
int MasterCount=0;
    
    
// Main Entry Point for app.
//-------------------------	
int main(int argc, char* argv[])
{
  printf("NxCore SampleApp Start.\n");
   
  // Set the state tape filename 
  strcpy(StateFileFilename,"STATESAVE.TMP");    
  
  if (LoadNxCore("NxCoreAPI.dll"))
  {        
    printf("\nThe application will process trades until 9:00 AM and then save the state,\n exit and reload from the state tape.\n\nPress  to continue.\n");
    getchar();    
    
    // If a tape filename was passed in command line argument,
    // call ProcessTape with that argument oherwise start in current tape. 
    if (argv[1])
      pfNxCoreProcessTape(argv[1],NULL,0,0,OnCoreCallback);
    else
      pfNxCoreProcessTape("",NULL,0,0,OnNxCoreCallback);    
    
    // The current process has exited and the state tape has been saved!
    printf("Tape state saved and process exited.\nPress  to Load the state and continue.\n");
    getchar();    
    
    // Reset this flag so we dont stop in the next call to ProcessTape
    SavedState=0;    
    
    // Process the tape from a state file
    pfNxCoreProcessTape(StateFileFilename,NULL,0,0,OnNxCoreCallback);    
    
    UnloadNxCore();
  }
    
  printf("NxCore SampleApp Stop.\n");    
  
  return 0;
}
    

// The NXCore Callback Function	
//-----------------------------
int __stdcall OnNxCoreCallback(const NxCoreSystem* pNxCoreSys,const NxCoreMessage* pNxCoreMsg)
{    
  
  // If this flag is set, NxCore has saved the state. Note that from the time
  // the NxCORESTATUS_SAVING_STATE message was received, the function issued
  // one moreNxCALLBACKRETURN_CONTINUE, and the next time in we will exit, thus
  // allowing NxCore to properly save the state and to exit automatically.
  if (SavedState)
    return NxCALLBACKRETURN_STOP;
        
  
  switch (pNxCoreMsg->MessageType)
  {
    
    // NxCore Symbol Spin Message 
    // NOTE: (will NOT be generated when NxCore is started from a state tape)
    case NxMSG_SYMBOLSPIN:    
	 
       // Set UserData1
	 pNxCoreMsg->coreHeader.pnxStringSymbol->UserData1=MasterCount++;	     
	 break;    
    
    // NxCore Status Message
    case NxMSG_STATUS:     
	 OnNxCoreStatus(pNxCoreSys, pNxCoreMsg);			 
	 break;    
    
    // NxCore Trade Message
    case NxMSG_TRADE:
	 OnNxCoreTrade(pNxCoreSys, pNxCoreMsg);
	 break;
  }         
  
  return NxCALLBACKRETURN_CONTINUE;  
}
   	

// OnNxCoreStatus: Function to handle NxCore Status messages
//----------------------------------------------------------
void OnNxCoreStatus(const NxCoreSystem* pNxCoreSys,const NxCoreMessage* pNxCoreMsg)
{   	
  
  // If a Minute has elapsed print the NxCore system time.
  // NOTE: The last time sent is 24:00:00. 24:00:00 is not actually
  // a real time value
  if ((pNxCoreSys->ClockUpdateInterval >= NxCLOCK_MINUTE) &&
      (pNxCoreSys->nxTime.Hour < 24))
  {
    printf("NxCore Time: %02d/%02d/%d  %02d:%02d:%02d\n",
	     pNxCoreSys->nxDate.Month,pNxCoreSys->nxDate.Day,pNxCoreSys->nxDate.Year,
             pNxCoreSys->nxTime.Hour,pNxCoreSys->nxTime.Minute,pNxCoreSys->nxTime.Second);
   
    
    // LOOK! If it's 9:00:00 AM SAVE THE CURRENT STATE TO A STATE FILE!
    if (pNxCoreSys->nxTime.Hour==9 && pNxCoreSys->nxTime.Minute==0 && pNxCoreSys->nxTime.Second==0)
    {
      
      // NOTE: This application will only work properly with the NxSAVESTATE_ONEPASS flag setting,
      // as we exit the tape exactly one pass after the file is to be saved. Should you modify this
      // flag the application will need to also be modified accordingly.
      pfNxCoreSaveState(StateFileFilename,NxSAVESTATE_ONEPASS);
    }
  }   	
  
  switch( pNxCoreSys->Status ) 
  {
    
   // NxCore will save the state this pass. Tell user and wait for keypress
    case NxCORESTATUS_SAVING_STATE:
         printf("\nNxCore Will Save the State Now!\nPress  to Continue.\n");	
         getchar();   	
         
         // Set the state saved flag
  	   SavedState=1;
         break;   	
     
   // NxCore Has been loaded from a state tape. Tell user and wait for keypress
    case NxCORESTATUS_LOADED_STATE:
         printf("NxCore Has Been Loaded From A State!\nPress  to continue processing trades.\n");	
	   getchar();
	   break;
  }
}
   	

// OnNxCoreTrade: Function to handle NxCore Trade messages.	
//--------------------------------------------------------------
void OnNxCoreTrade(const NxCoreSystem* pNxCoreSys, const NxCoreMessage* pNxCoreMsg)
{
  if (pNxCoreMsg->coreHeader.pnxStringSymbol->String[0]=='e')
  {    
    NxCoreTrade* Trade = ( NxCoreTrade*) &pNxCoreMsg->coreData.Trade;   	
    
    // Get the price and net change
    double Price = pfNxCorePriceToDouble(Trade->Price,Trade->PriceType);
    double NetChange = pfNxCorePriceToDouble(Trade->NetChange, Trade->PriceType);   	
    
    // Write out Symbol, Time, Price, NetChg, Size, Reporting Exg and UserData
    printf("Trade for Symbol: %s, Time: %02d:%02d:%02d  Price: %0.2f  NetChg: %0.2f  Size: %d  Exchg: %d  UserData1: %d\n",
		   pNxCoreMsg->coreHeader.pnxStringSymbol->String,
		   pNxCoreMsg->coreHeader.nxExgTimestamp.Hour,pNxCoreMsg->coreHeader.nxExgTimestamp.Minute,pNxCoreMsg->coreHeader.nxExgTimestamp.Second,
		   Price,NetChange,Trade->Size,
		   pNxCoreMsg->coreHeader.ReportingExg,
		   pNxCoreMsg->coreHeader.pnxStringSymbol->UserData1);
  }
}