Welcome to NxCore
Welcome to the NxCore API. (NxCore is pronounced "N" Core or "'Encore").
The NxCore API is the fastest possible access to the ultra-low latency NxCore Financial Data Stream (NxCore Feed). There are 3 basic components that were installed on your computer.
1. NxCoreAccess
NxCoreAccess is a program that maintains a connection to the NxCore Feed Servers and is responsible for bringing the NxCore Feed into your computer. It also can download historic NxCore Feed data (one or more days older) in the background. You can run it as an application or as a service.
NxCoreAccess is the first program you will see after running NxInstall. After the login prompt, it will display a chart showing the data flowing from NxCore Servers to your computer in one-second intervals. There are 4 main views which are accessed from tiny buttons in the upper left-hand corner labelled C (Chart), T (Tape files), L (Event Log), and what looks like the equals sign (Compact Mode). There is a menu which appears when you click on the "N" in NxCoreAccess on the Title Bar -- the "N" is in the top left corner where the system menu is commonly found (it is the system menu).
NxCoreAccess was designed to run all the time - 24 hours a day, 7 days a week, 365 days a year - while requiring very little input from you. When you click on the upper right "x", which typically closes an application, NxCoreAccess disappears from your desktop and is visible only in the tray. NxCoreAccess comes with NxCoreAccessService, whose function is to ensure NxCoreAccess continues running. In order to terminate NxCoreAccess completely, both the application and the service must be killed.
NxCoreAccess v 2.8.64 was introduced in Q4 of 2015 for subscribers receiving end of day tapes, and will replace any downloader applications for these suscribers by Q1 of 2016. While it does take advantage NxCoreAccess's features, NxCoreAccess v 2.8.64 will not download anything during market hours. Rather, it will begin streaming download of the previous day's tape shortly after midnight.
NxCore Tape
Each day, NxCoreAccess produces one file on your computer, an NxCore tape, which contains all the quotes and trades (and everything else) that was sent in the NxCore Feed for that day, beginning at exactly 12:00am and ending at 12:00pm Eastern Time. These tape files are stored in the [X]:\NxCoreData directories for the one or more drives (double click the 2nd cell from the right on the top bar in NxCoreAccess to configure drives). The naming convention for NxCore Tapes is: YYYYMMDD.ABCDE.nxc.
If the connection between NxCoreAccess and the NxCore Servers is broken due to power outage, network outage, or a hardware failure on your machine, NxCoreAccess will always reconnect so that the NxCore Tape File(s) will not have any missing or duplicate or unreadable data. There are several layers of error checking that NxCoreAccess uses when identifying the proper sequencing for each NxCore Tape. During development, it was routine to never close the NxCore Tape files properly -- simply shutting down the application from the debugger, or from power outages (no UPS), or even intentional abuse to the file. In all cases, the NxCore Tape files were automatically repaired and verified to be identical to the original NxCore Tape files as sent from the NxCore Servers. This means you can let NxCoreAccess worry about the details in delivering error-free and complete trade and quote streams.
The contents of a NxCore Tape depends on exchange permissions. Every completed tape of the same name is exactly identical to any other completed tape of the same name, worldwide.
NxCoreAccess connects to port 55555 (five 5s) on an NxCore Server. Make sure you allow all servers for your NxCore Feed Group.
| Full OPRA & Equity+ | Equity+ | OPRA NBBO & Equity+ |
|---|---|---|
| fg1s1.nanex.net | fg2s1.nanex.net | fg4s1.nanex.net |
| fg1s2.nanex.net | fg2s2.nanex.net | fg4s2.nanex.net |
| fg1s3.nanex.net | fg2s3.nanex.net | fg4s3.nanex.net |
| fg2s4.nanex.net | fg4s4.nanex.net | fg2s5.nanex.net | fg4s5.nanex.net | fg2s6.nanex.net |
2. NxCoreAPI.dll
The NxCoreAPI.dll is a Dynamic Link Library that your application dynamically links with to access the NxCore Feed.
Don't worry if you have not dynamically linked to DLLs before; it is a simple procedure and there are plently of working code examples so you can just cut and paste a few lines of code.
The methods you use from NxCoreAPI.dll to process the NxCore Feed are collectively referred to as the NxCore API (application program interface).
There are less than 20 functions in NxCoreAPI.dll, and only one function is required --
NxCoreProcessTape which starts the data streaming into your application.
One of the parameters you pass NxCoreProcessTape is the address of one of your functions (the callback function) which will be called for each trade or quote update.
Your callback function is passed two formal parameters which are pointers to fixed data structures containing the quote and trade data.
Your job as an application programmer is copy the data members in which you are interested.
Note:
NxCoreAPI.dll connects to NxCoreAccess's internal memory buffers to process the real-time data stream. It can also process any NxCore tape file without the need for NxCore Access.
When NxCoreAPI.dll is connected to NxCoreAccess and processing the real-time stream, you can exit NxCore Access, and the NxCoreAPI.dll will patiently wait for NxCoreAccess to return at the exact last message it processed -- so that when NxCoreAccess is started again, your callback function will receive the exact same next record as if NxCore Access was never stopped. This means you can upgrade to new versions of NxCore Access without having to restart or resynchronize in any way with the NxCore Feed. NxCore Tapes are deterministic: you can replay an NxCore Tape any time, and your call back function will always be called the exact same number of times with the exact same information. NxCore tapes with the same date and permission set will be identical on any two machines in the world. This fact will save you considerable time tracking down and reproducing problems even if your development team is remote.
You can run more than one instance of NxCoreAPI.dll simultaneously -- from the same process, or from different processes. Each NxCoreAPI instance can process the real-time NxCore Tape or a previously saved NxCore Tape. Each instance of NxCoreAPI is safely isolated from other instances so that there are practically zero contention issues. Typically, 2 instances of NxCoreAPI.dll can process 2 streams using the full 100% of CPU on a dual-processor machine, and 4 instances can simultaneously process 4 streams using the full 100% of a quad-processor machine because there is zero contention between the NxCoreAPI.dll instances. Each instance has its own private memory pool which means if a process or thread has a bug that corrupts its memory structures, the other instances will continue running unharmed.
NxCore Access can simultaneously service multiple threads and multiple processes connected to its real-time memory buffers. It is structured so that if one thread or process misbehaves, or is stopped in a debugger for a long time, the other processes and threads will continue to run uninterrupted.
3. NxCoreAPI Sample applications and NxCoreAPIViewer.exe.
There are several sample applications with source code that illustrate basic concepts. You will find these applications in your Program Files directory which is typically:
C:\Program Files\Nanex\NxCoreAPI\Doc
Tip: An easy way to navigate to NxCore files is to click on the Diagnostics toolbar button within NxCoreAccess, and select one of the menu items that begins with the text Explore. For example, Explore Program Directory will open Windows Explorer with the path set to the NxCore programs.
Once you are receiving data in NxCore Access, click the first toolbar button labelled Run Viewer to launch the program NxCoreAPIViewer which is a collection of interactive programs that display the NxCore data fields available to your callback function. The main display of NxCoreAPIViewer presents a tree view of the actual "C" structure members of the parameters passed to your callback function.
After viewing the data in NxCoreAPIViewer, open the Visual C++ project file for SampleOne. SampleOne is a very simple program including "C" source that illustrates the minimal elements an application needs to process the NxCore Feed. Stripped of frills, it makes the basics easier to see and understand and offers a good starting point.
Ignore the Category message types until you master the basic quote (ExgQuotes and MMQuotes) and Trade updates. You do not need any information the Category messages for processing Quote and Trade messages.
Miscellaneous Notes
Warning:
NxCore sample code updates can and will overwrite or even delete files in directories beneath the Nanex root: C:\Program Files\Nanex. Therefore, it is strongly recommended that you always make a copy of whatever files you are working with into a seperate directory. You can always install older versions, but that won't restore any files or changes you may have added to C:\Program Files\Nanex and below.
NxCoreAccess, for example, copies NxCoreAPIViewer.exe and NxCoreAPI.dll to its working directory when you select "Open NxCoreAPI Viewer" from the system menu (you can see the working directory path in the LogView window of NxCoreAccess). This way, there are no collisions when new versions of NxCoreAPI components are available.
NxCore Versioning
NxCore uses a common Major.Minor.Build versioning system. The build date and time are also included with the version number. NxCoreAPI will archive the current version in a seperate directory whenever the Major or Minor version number changes. This allows you to easily compare changes to any include files or text files, and also allows you to run new versions alongside of older version giving you confidence and control over the upgrade schedule.
For example, when the current version of NxCoreAPI changed from 2.5 to 2.6, a new versioned subdirectory was created:
C:\Program Files\Nanex\NxCoreAPI\Version\2.6
The new 2.6 files were copied into this new version subdirectory directly and also replaced any files in the main NxCoreAPI directory:
C:\Program Files\Nanex\NxCoreAPI\
The 2.5 version files would still be available in the version subdirectory that was created when it was installed:
C:\Program Files\Nanex\NxCoreAPI\Version\2.5
So the current version is always found in 2 places: the main NxCoreAPI directory, and its version subdirectory. Use the main NxCoreAPI directory if you always want to process the latest version, and use a version subdirectory to control which version of NxCoreAPI.dll you use.
Note: NxCoreAPI for the most part has remained unchanged and consistent over the years. Changes are almost exclusively found in new category messages and fields.
NxCore Philosophy
The underlying philosophy behind NxCore is
"to be as simple as possible, delivering the richest data possible, using the smallest memory/CPU footprint possible, but without in any way compromising the basic fundamental task of delivering quote and trade updates as fast as the state of the art allows."
Each data field in the memory blocks passed to your callback function was selected based on this philosophy. You will discover many fields available with each quote/trade update in NxCore Feed that you will not find in any other feed. An understandable initial reaction is that this extra information impacts update performance and NxCoreAPI would be even faster if it had to update less. But this is not the case: the organization of the data and each fields impact on speed (latency) was exhaustively scrutinized before making the selection for inclusion. Since these fields are already available with each update, just sitting in memory for immediate use, you will find there is much less work that you have do to (and much less CPU time you need to consume), and the overall effect with be significant faster record processing.
To illustrate this, for example, a common initial task in any feed processing application is mapping the feed symbol strings to your data structures. Done improperly, this will consume a lot of CPU time (imagine a string compare for each update -- at 5 million updates/second). Each quote/trade update in NxCoreAPI offers several fields in memory to assist various methods of FAST symbol mapping. Two of these fields are unique slots in each symbol space that you can fill with any 32-bit value you wish, and will be preserved automatically and appear each time that symbol updates. One incredibly simple, yet effective, symbol mapping method would be to store your data stucture for each symbol in one of those slots which would reduce your indexing time cost to nil.
NxCore Server Components
NxCore Feed Processors
The NxCore Feed Processors reside on the NxCore Network that reads in real-time data streams, performs analysis, and then outputs the NxCore Feed for distribution to the NxCore Feed Servers.
NxCore Feed Server
The NxCore Feed Servers distribute the NxCore Feed to NxCoreAccess applications running on customer machines. NxCore Feed Servers are divided into Full, EquityPlus and OPRA NBBO NxCore Feed Groups that span multiple geographic areas and internet service providers to ensure optimized connectivity and speed.