States Files and Memory in NxCore
| As NxCore is a whole market-feed processing system, it must process tapes
from the start of the day in order to properly populate and maintain internal
data structures through all corrections and exchange messages. However, NxCore
is capable of saving it's internal state at any time and restarting from that
point using the saved state file. For instance, if an application crashes or is
shut down unexpectedly, the application can restart at the time the last state
file was saved and begin processing trades and quotes from that point forward.
One of the caveats to this approach is insuring a consistent area of memory for
NxCore to process data from. A state file is quite simply, a dump of the memory area NxCore is processing data in. A critical point to consider when loading a state file is that the file will be loaded into the same memory area it was created in. If the state file cannot be loaded into the same memory area, the load will fail. In order to insure a successful state file load the programmer must also insure the state file can be loaded into the same memory area it was created in. Another point to consider is that all Windows applications use their own virtual address space that cannot be interfered with by another application, nor will it interfere with other applications. This means that any particular address space is local to the specific application, allowing safe memory allocation at specific addresses regardless of whether other applications have consumed memory using the same virtual address. Consider the following sequence of events (a common one many NxCore users experience) in which the load of a state file fails:
This scenario can easily be avoided by specifying the starting memory address NxCore uses when calling ProcessTape, and using an address that is known to be consistently available in the application. As an example, consider the following where it has been determined memory address 0x2000000 is consistently available with enough contigous free memory for NxCore to load into:.
Knowing What Memory Address to Use In a production environment, applications should start up and consume memory identically as the previous load. Given this and the fact that all Windows applications use their own virtual address space, determining a safe address to use when starting NxCore is a simple matter of scanning memory once the application is fully loaded to determine available address's and consistently use that address. If an application uses different start-up sequences (that could depend on various factors), each sequence should be run and address's scanned to determine an available area in all possible start-up sequences. If state files are to be shared across multiple machines, then all machines using the application should be scanned for a memory address that is available on all systems. How Much Memory is Needed The exact amount of contiguous memory needed to load a state file is (160*1024*1024) bytes, or 160 MB. Memory Address Scan, Run and Lock Simulation Application To best explain reserving memory locations for NxCore/ProcessTape to run in, we have written a simple application (code is available to all NxCore customers) to demonstrate the principles:  Click Here for a detailed tutorial on using the application. |