JTOOLS


Hello and welcome. My name is Jeffrey Donovan and I am a senior software engineer at Nanex, Inc.. I have been with Nanex since 2009 and previous to that, I was a Nanex customer writing commercial applications based on NxCore, most of which are still in use today.

During the course of developing with NxCore, I have written dozens of stand-alone applications, both small and large (and am continually writing more). Many of these are used on a daily basis by us at Nanex to quickly tackle customer support issues, while others are applications more in line with real-time use of NxCore (such as quote monitors, stock charting, etc). Beginning in May/June of 2012 I put some final touches on a few of these apps and designed each to work under an identical framework. I am making the applications and the full source code available to all NxCore developers and if you would like to receive one or all of them please email support@nanex.net and ask for the 'JTools' library (yes JTools stands for Jeff Tools). Please include your name and current Nanex account details. I will be making more of these applications available as time permits.

The applications were designed to be robust, yet simple and easy to understand by any intermediate level programmer (or beginning programmer with a little extra effort). I am a believer in minimal abstraction and the code should reflect this fact. As a Nanex customer you are free to modify and use the code in your own applications as you see fit and should you make improvements to these apps, I (we) would love to see them!



Detailed notes about design philosophy, naming conventions, file locations, compiler settings and misc. notes.

Issues when running for the first time - No NxAccess

State Files - A Valuable Tool


Volume I - Support Utilities, Data Extraction

JTools Shell Application Date Added: 07/04/2012 Last Date Modified: 07/04/2012

A Generic Shell Application for running NxCore. This application features a completely detachable GUI, logging, real time simulation, delayed simulation and a variety of other options. This is the application I usually start with to build any new NxCore based application. It has served me very well.

Along with the Generic Shell Application are included two spin-off projects to demonstrate two core principles:

JTools_ShellApp_Console Date Added: 07/04/2012 Last Date Modified: 07/04/2012
Demonstrates the detached GUI and how easy it is to un-attach it or pump another type of GUI (such as a console application):

JTools_ShellApp_Dual Date Added: 07/04/2012 Last Date Modified: 07/04/2012
Demonstrates how the shell application's interface uses/modifies only data passed though the NxCore UserData pointer and as such, spinning multiple instances of the NxCore within a single application becomes trivial.



JTools Category Tracker Date Added: 07/04/2012 Last Date Modified: 09/06/2013

The Category Viewer provides easy viewing of all category messages as they occur. Numerous filters and options make drilling down for a specific message easy. We use the application on a daily basis to help in solving/debugging client problems with category messages.



JTools Message Tracker Date Added: 07/04/2012 Last Date Modified: 07/15/2015

The Message Tracker provides easy viewing of all messages as they occur, with extreme detail on Trade, ExgQuote and MMQuote messages. Numerous filters and options make drilling down for a specific message easy. As with the Category Viewer, we use the application on a daily basis to help in solving/debugging client problems with several message types.



JTools MSG Rate / Performance Monitor Date Added: 07/04/2012 Last Date Modified: 07/04/2012

The MSG Rate / Performance Monitor measures throughput of messages through NxCore/end user application. It can be extremely valuable in determining what historical throughput should be expected on any machine, as well as monitoring real time performance. Performance is displayed in a chart similar to the bandwidth chart in NxAccess.




JTools State File Generator Date Added: 07/04/2012 Last Date Modified: 07/04/2012

The State Writer Application is a revision of the JDStateWriter application, which I made available to users some time ago. Since the initial version however, I have modified the application for a numbers of various tasks as well as improve the code in general. I have recently been converted it to the JTools framework and I have included every option I have ever needed with regards to automatic NxCore state file generation.



JTools Time & Sales State File Reader Date Added: 07/04/2012 Last Date Modified: 07/04/2012

The Time and Sales State Reader Application is a working time and sales viewer that demonstrates the use of NxCore State Files and the JTools_StateFileGenerator application. Instantly jump from time frame to time frame over one or multiple dates in NxCore.




JTools Time & Sales File Generator Date Added: 07/04/2012 Last Date Modified: 08/29/2012

The JTools_TSFileGenerator is a command line console application driven by the arguments passed into it. It allows you to dump time and sales reports for any single symbol in almost any fashion you desire. Every field available for NxCore Trade, ExgQuote and MMQuote messages is offered as possible output. Fields generated and field order are determined by the usage strings passed into the application arguments (see below for a complete list of possible fields). It will dump the output to the console (if specified to) and to a text file, the name of which is also passed in as an argument.
Also included are 2 small projects that provide a GUI interface to launch the TS File Generator with.




JTools EOD File Generator Date Added: 07/04/2012 Last Date Modified: 07/04/2012

The JTools_EODFileGenerator allows you to dump summary trade information for the entire universe of symbols in almost any fashion you desire. The system gathers information from NxCore State Trades. As such it can be run from state files generated just before or at the desired time. Every field available for NxCoreStateTrade is offered as possible output. Fields generated and field order, as well as symbol types (IE Equities, Futures, Indexes, etc) are determined by the usage strings. It will dump the output to the console (if specified to) and to a dated-timed text file(s).

The application comes in 2 flavors - a console application driven by command line arguments passed into it and a GUI driven application with all setting fields in the GUI.




JTools Interval File Generator Date Added: 08/29/2012 Last Date Modified: 04/12/2015

The Interval File Generator is very similar to the EOD File Generator but takes the data generation one step further by also providing true interval data. That is to say that if the timing interval is set to 1 then the values generated are for 1 specific minute in time. Along with true interval data the application can also generate timed EOD data, where data represents the composite values for the day.




JTools Interval File Generator GUI Date Added: 08/29/2012 Last Date Modified: 04/12/2015

Just like the standard JTools_IntervalGenerator console application, the JTools_IntervalFileGeneratorGUI allows you to dump trade information for the entire universe of symbols in almost any fashion you desire. The system gathers information from NxCore Trade messages for interval data, and State Trades for EOD data. The application can be run from state files generated just before or at the desired time. Fields generated and field order, as well as symbol types (IE Equities, Futures, Indexes, etc) are determined by the usage strings (see below for a complete list of possible fields).

Unlike the console based Interval Generator, the JTools Interval Generator GUI application also combines the functionality of the JTools Batch Processor to make processing data which spans days, months or years trivial.




JTools Interval File Generator for Futures / Future Options Date Added: 05/12/2014 Last Date Modified: 04/13/2015

The Interval File Generator for Futures and Future Options is very similar to the Standard Interval File Generator but contains some modifications and special features.

The following interval data can be produced with the The JTools_IntervalFileGenerator for Futures:
  • One specific future.
  • One specific future and all underlying future options.
  • All futures belonging to a specific root symbol.
  • All futures belonging to a specific root symbol plus all underlying future options.



JTools Interval File Generator for Stocks / Stock Options Date Added: 03/18/2015 Last Date Modified: 03/18/2015

The Interval File Generator for Stocks and Stock Options is very similar to the Standard Interval File Generator but contains some modifications and special features.

The following interval data can be produced with the The JTools_IntervalFileGenerator for Stocks/Stock Options:
  • One specific stock, all stocks or a specific alpha series of stocks.
  • One specific stock and all underlying stock options.
  • Specific alpha series of stocks and all underlying stock options.



JTools Table Viewer Date Added: 09/27/2012 Last Date Modified: 09/27/2012

With the exception of the Generic Shell Application, the Table Viewer application is the simplest of all JTools applications. The Table Viewer simply retrieves the string definitions for each NxCore table. These tables include (but are not limited to) exchange codes and names, trade conditions, quote conditions, etc. By using the tables a developer has access to human readable string definition for all NxCore tables without the need to hard-code them in. More importantly, the table definitions are updated automatically each day so there is never a need to update them manually.




JTools Symbol Change Analyzer Date Added: 02/20/2013 Last Date Modified: 03/01/2013

As NxCore is a whole market feed, many Nanex customers will build very sophisticated applications using it. These may include Bar Data servers, Streaming Data servers, Tick Data servers and/or a host of other applications designed to maintain and deliver data to their own customers. The majority of these applications will consist of proprietary databases that are fed with the NxCore stream. In this regard, properly tracking new symbols, or changes to existing symbols becomes critically important.

The Symbol Change Analyzer was developed to aid in this monitoring. It tracks all relevant information regarding symbol additions and changes and has a variety of mechanisms for flagging potential errors.




JTools Halt Monitor Date Added: 08/19/2013 Last Date Modified: 08/19/2013

The Halt Monitor is a simple application that monitors halts in all issues throughout the trading day. The Halt Monitor listens to NxCore category 63 (halt) and will track the current status of any issue that has been halted, display the reason(s) for the halt and track the number of times an issue has been halted.




JTools FOS Lookup Zombie Date Added: 10/29/2014 Last Date Modified: 10/29/2014

Like equities, futures may also have options associated with them. Unlike equities, futures may also have spreads associated with them. While NxCore has built in mechanisms to identify a root symbol for a specific equity option (or option chain), there is no such mechanism for future options and spreads. Futhermore, future option symbols may be different than the root symbol prefix of the option (much like equity options prior to 2010 and OSI, the Option Symbol Initiative). As such one of the most common questions from users trading futures is how to properly associate the future options and spreads to a specific underlying future.

The FOS (Futures, future Options, future Speads) was developed to demonstrate these concepts but goes much futher. It is one of the more complex JTools components and the user/developer will be well served to study the code closely. Once understood it will be possible to build complex applications using the existing code.



Volume II - Trade/Quote Monitors, Charting, Market Analysis

JTools HFT Quote Montage Date Added: 07/04/2012 Last Date Modified: 05/09/2014

The Quote Montage HFT Application is one I initially wrote to test a new server and at the same time, we wanted a new tool for viewing specific quoting stats in any single stock (specifically when we would see high speed algos running in particular issues). I've recently converted it over to the JTools/NxCore framework and it is humming along better than ever. The application demonstrates several key concepts related to performance oriented, real world NxCore/GUI trading systems.



JTools Futures Depth Montage Date Added: 07/04/2012 Last Date Modified: 07/04/2012

The Futures Depth Montage is another application I initially wrote to test a new server and at the same time, provide a new tool to view futures depth and depth stats. I have recently converted it over to the JTools/NxCore framework. The Futures Depth Montage demonstrates several key concepts related to both NxCore/GUI systems intended for real world use.




JTools One Second Charter Date Added: 07/10/2012 Last Date Modified: 05/09/2014

Bar charting is perhaps one of the most common ways for humans to look at stock data (or futures, etc). Bar Charts display price activity in discrete intervals which can be anything from one day to several milliseconds.

The One Second Bar Charter is a simple application that plots data in one second time increments. Both price and volume are plotted and the price may be plotted as traditional bars or as candlesticks




JTools Vwapper Date Added: 07/11/2012 Last Date Modified: 05/09/2014

The Vwapper Application is an extension of the One Second Charter Application that also charts VWAP (Volume Weighted Average Price) and VWAP vs Price difference.

Because the most common VWAP calculation requires VWAP to be calculated from the start of trading to the end of the trading day, The Vwapper Application calculates and maintains VWAP numbers for every issue in the system and for every trade received. As such, when you bring up new issues, the VWAP number is current and the charting will reflect this. It is yet another example of the 'whole market analysis' one can perform with NxCore. This is not only applicable to this charting app, but any application that uses VWAP.




JTools Top Ten Blaster Date Added: 07/10/2012 Last Date Modified: 07/10/2012

The Top Ten Blaster application calculates and maintains 5 top ten lists for every exchange available. The lists are then separated by Equities, Futures and Indexes, depending on what the exchange offers. It is a perfect example of the 'whole market analysis' one can perform with NxCore.

The application also calculates and maintains the number of advancing, declining and unchanged issues for every exchange available. Like the top ten Lists, they are separated by Equities, Futures and Indexes, depending on what the exchange offers.



JTools Pulsar Monitor Date Added: 07/16/2012 Last Date Modified: 09/01/2012

The Pulsar Monitor demonstrates the concept of 'pulsed' quotes and trades (a concept used in practice by many). In today's environment, even non liquid stocks can see quote rates exceeding 5000+ quotes per second, which can place an enormous load of both server and client software. As opposed to sending out every quote and trade, pulsed systems send out quotes and trades on a regular time interval, which usually ranges anywhere from 100 milliseconds to 1 second. In doing this, considerable bandwidth in both delivery and processing of data is saved.

The concept is demonstrated by pulsing quotes/trades to a GUI that would be the type of system appropriate for pulsed data, which is a basic monitor containing user defined symbols, updating on a real time basis with composite trade and quote data.




JTools Option Chain Viewer Date Added: 10/05/2012 Last Date Modified: 10/05/2012

If you receive options though NxCore then you most likely have a keen interest in building Option Chains. The following application shows you how to do just that, plus a little more. Option Chains represent all the options available for an underlying equity. Typically when used for displays the options are separated by puts and calls and are sorted according to strike price and time to expiration. Because some equities may have thousands of options, most applications (including this one) will allow you to filter options on various criteria.




JTools FOS Live Chains Date Added: 10/29/2014 Last Date Modified: 10/29/2014


The FOS Live Chains application is very similar to the FOS Lookup Zombie application, taking it one step further to demonstrate futures, future option chains and future spread chains updating and populating with real time data. If you are not familar with the FOS System please see the documentation for the FOS Lookup Zombie application.



Volume III - Market Stress, Market Phenomena, Research

JTools Alpha Channel Heat Mapper Date Added: 07/20/2012 Last Date Modified: 07/20/2012

During our Flash Crash analysis, we were constantly looking for new ways to visualize the information we were looking at. One of the focus's of the analysis was on delays and saturation of CQS/UQDF due to extremely high quote traffic. Because the feeds from each have Channels, or Lines, which represent stocks by alphabet, we wanted a tool to show us how each channel was effected to a given circumstance. I developed a charting system that would show intensity of any given alpha channel by color.


JTools One Sec Exg Plotter Date Added: 08/05/2012 Last Date Modified: 08/21/2012

The One Second Exchange plotter charts every exchange independently for trades, bid quotes and ask quotes in one second intervals. It was one of the primary tools we used in doing our initial Flash Crash analysis to visually confirm data. This application has been refined and revised many times since we began our research and many new items have been added.


JTools QTSequencer Date Added: 08/05/2012 Last Date Modified: 09/18/2013

The QTSequencer (Quote-Trade Sequencer) is an application that charts every quote and trade (independently for each exchange) sequentially with no data loss or compression. Originally developed for our Flash Crash Analysis it remains a tool we use each and every day. This application has been refined and revised many times since we began our research and many new items have been added.


JTools HotPotato Date Added: 08/20/2012 Last Date Modified: 08/20/2012

Using tools such as the QTSequencer, one can easily visualize stock activity on a sub-second (tick) level. However, randomly poking about different times in various stocks will yield little valuable results, what is needed is something to screen through the universe of stocks in real time and display the results.

The HotPotato application is a conglomeration of three filters I continue to use on a daily basis - High Quote Rates, Wide NBBO Spreads and Explode/Implode (mini-crash) filters.



Volume IV - Nx3D - 3D Visualization

JTools 3D Depth Mapper - Static Date Added: 12/08/2012 Last Date Modified: 12/08/2012

The 3D Depth Mapper application takes futures depth into another dimension. Developed at Nanex to examine liquidity expansion and contraction in the futures markets to see potential patterns that may not be obvious on a 2D display. It works in one second intervals and is is one of our more advanced (and unique) real time charting applications. The 3D Depth mapper does not simply provide a static image for viewing but lets you navigate through the image, allowing deeper investigation of individual components.


JTools 3D Depth Mapper - RealTime Date Added: 12/08/2012 Last Date Modified: 12/08/2012

The 3D Depth Mapper application takes futures depth into another dimension. Developed at Nanex to examine liquidity expansion and contraction in the futures markets to see potential patterns that may not be obvious on a 2D display. It works in one second intervals and is is one of our more advanced (and unique) real time charting applications. The 3D Depth mapper does not simply provide a static image for viewing but lets you navigate through the image, allowing deeper investigation of individual components.


JTools 3D QTSequencer - Static Date Added: 12/08/2012 Last Date Modified: 12/08/2012

The 3D Quote/Trade Sequencer is an extension of the JTools_QTSequencer application, bringing a new dimension to examining sequential quotes and trades. Developed at Nanex to examine potential algorithmic patterns that may not be obvious on a 2D display. Like the 2D version of the sequencer, the 3D version plots every quote and trade by every reporting exchange sequencially as they occur.


JTools 3D QTSequencer - RealTime Date Added: 12/08/2012 Last Date Modified: 12/08/2012

The 3D Quote/Trade Sequencer is an extension of the JTools_QTSequencer application, bringing a new dimension to examining sequential quotes and trades. Developed at Nanex to examine potential algorithmic patterns that may not be obvious on a 2D display. Like the 2D version of the sequencer, the 3D version plots every quote and trade by every reporting exchange sequencially as they occur.



Volume V - Odds & Ends

JTools ASCII Compressor Date Added: 02/05/2013 Last Date Modified: 02/05/2013

The ASCII compression utility is a very small application that does not use NxCore and is not based on the JTools_ShellApplication framework. Rather, it is a simple application to demonstrate simple compression (up to 50%) and decompression of ASCII data (which can be implemented on any platform) with minimal performance impact. This can be very useful to clients who redistribute financial data in ASCII format.


JTools Batch Processor Date Added: 02/20/2013 Last Date Modified: 02/20/2013

Because of the in-depth historical files NxCore provides, there are several times when the developer may want to run an application over many tapes. Possibly for analysis, possibly to populate historical databases.

The Batch Processor application is a very simple modification of the Generic Shell App. It takes a start date, stop date (and a few other parameters) and processes through the dates, stopping only when the last historical file has been processed. With very little effort, any one of the appropriate JTools applications could be modified to batch process historical files using the code provided with the Batch Processor application.