Table of Contents


States Files and Memory in NxCore

As NxCore is a whole market-feed processing system, it must process tapes from the start of the day in order to properly populate and maintain internal data structures through all corrections and exchange messages. However, NxCore is capable of saving it's internal state at any time and restarting from that point using the saved state file. For instance, if an application crashes or is shut down unexpectedly, the application can restart at the time the last state file was saved and begin processing trades and quotes from that point forward. One of the caveats to this approach is insuring a consistent area of memory for NxCore to process data from.

A state file is quite simply, a dump of the memory area NxCore is processing data in. A critical point to consider when loading a state file is that the file will be loaded into the same memory area it was created in. If the state file cannot be loaded into the same memory area, the load will fail. In order to insure a successful state file load the programmer must also insure the state file can be loaded into the same memory area it was created in.

Another point to consider is that all Windows applications use their own virtual address space that cannot be interfered with by another application, nor will it interfere with other applications. This means that any particular address space is local to the specific application, allowing safe memory allocation at specific addresses regardless of whether other applications have consumed memory using the same virtual address.


Consider the following sequence of events (a common one many NxCore users experience) in which the load of a state file fails:
  1. ProcessTape is called without specifying a starting memory address in the arguments. It finds memory location 0x1000000 is available begins to use memory at that address for processing.

  2. During execution of the tape, a state file is created and saved.

  3. The program is stopped in the middle of the day and an attempt to restart using the last saved state file is made.

  4. During the restart, the developer changes the order of loading specific elements in the application that consume memory. In this case one of the elements allocates memory at location 0x1000000 before NxCore is called to start from the previously saved state file..

  5. As NxCore cannot use memory location 0x1000000, the load of the state file fails (ProcessTape returning -5).

This scenario can easily be avoided by specifying the starting memory address NxCore uses when calling ProcessTape, and using an address that is known to be consistently available in the application. As an example, consider the following where it has been determined memory address 0x2000000 is consistently available with enough contigous free memory for NxCore to load into:.

  1. ProcessTape is called and memory address 0x2000000 is specified in the start-up arguments.

  2. During execution of the tape, a state file is created and saved.

  3. The program is stopped in the middle of the day and an attempt to restart using the last saved state file is made.

  4. During the restart, ProcessTape is called to load the last state file saved. As memory address 0x2000000 is available and is the same address ProcessTape used when creating the state file, the state file loads successfully and the tape continues to run from that point forward..


Knowing What Memory Address to Use

In a production environment, applications should start up and consume memory identically as the previous load. Given this and the fact that all Windows applications use their own virtual address space, determining a safe address to use when starting NxCore is a simple matter of scanning memory once the application is fully loaded to determine available address's and consistently use that address. If an application uses different start-up sequences (that could depend on various factors), each sequence should be run and address's scanned to determine an available area in all possible start-up sequences. If state files are to be shared across multiple machines, then all machines using the application should be scanned for a memory address that is available on all systems.

How Much Memory is Needed

The exact amount of contiguous memory needed to load a state file is (160*1024*1024) bytes, or 160 MB.

Memory Address Scan, Run and Lock Simulation Application

To best explain reserving memory locations for NxCore/ProcessTape to run in, we have written a simple application (code is available to all NxCore customers) to demonstrate the principles:





Click Here for a detailed tutorial on using the application.