Systematic Approaches Apply Method to the Madness
Copyright 2000 Barclay Trading Group, LTD.
Terms, Conditions and Trademarks Apply.
Many academicians have taken the position that the futures markets are highly efficient with respect to price discovery. If you accept this hypothesis, you've taken a step towards the conclusion that it's not possible to make money trading the futures markets.
Our research and experience do not support this conclusion. The Barclay CTA Index has been profitable in 12 of the past 13 years. The compound annual return for the index over this time period is 19.57%.
Of the more than 550 managed programs in the Barclay CTA Database, almost two-thirds could be categorized as having a systematic, as opposed to judgmental or discretionary approach to trading. Six of the ten largest firms, as measured by amount of money under management, are systems traders.
In order to understand how a system approach that takes human judgment out of the equation is able to deal with what is considered by many to be a random and unpredictable market, we invited a distinguished panel of CTAs to comment on the subject. Our panel includes:
Barbara S. Dixon, Spackenkill Trading Corp. Ms. Dixon is the president of Spackenkill Trading Corp. Ms. Dixon began trading in 1973 and is a senior vice-president at Smith Barney.
William Eckhardt, Eckhardt Trading Co., Inc. Mr. Eckhardt is the president of Eckhardt Trading Company, Inc. Mr. Eckhardt began trading in 1974 after four years of doctoral research at the University of Chicago in mathematical logic.
Stig E. Ostgaard, Sjo, Inc. Mr. Ostgaard is a vice-president and director of research for Sjo, Inc., a Chicago-based CTA firm which was founded in 1982. Sjo currently manages approximately $137 million in three programs.
Rutherford R. Romaine, RXR, Inc. Mr. Romaine is a director and executive vice-president of RXR, Inc. RXR was founded in 1983 and is one of the ten largest CTA firms.
Q: Systematic approaches are based on extensive analysis of historical data. How is your systematic approach able to deal with unprecedented political and/or economic changes?
Dixon: The historical data that systems traders use to construct and test their trading rules contain then-unprecedented political and economic events which impacted prices. In general, the data lack labels such as January 31, 1990: Start of Gulf War or September 16, 1992: Collapse of EMS. The point of systems trading is that the impact of such events is always reflected in the price (whatever the market) and that systems traders need not struggle to react when such events actually occur; they simply follow their rules. As a risk control matter some systems designers may incorporate overrides to their rules designed to acknowledge such events, but I would view such actions as more discretionary than systematic.
Eckhardt: First, a statistical digression: small sample statistics in which delicate techniques are used to wrench marginal significance from sparse data have no place in futures trading. So what is a sufficiently large sample? In normal distribution dominated classical statistics, the answer is "more than 30". In view of the pathology of futures distributions, the correct answer for our field is more than 100.
Now how many unprecedented episodes--say on the order of magnitude of the Gulf War or the '87 stock market crash--are you going to have in your data? Maybe half a dozen. So historical analysis is doomed to statistical insignificance on this subject. This is the stuff that system overrides are made of. In these rare cases, there is no substitute for a human pilot.
Ostgaard: Unprecedented changes can be of two types: the gradual and the sudden. The gradual changes--an unfolding inflation or a consistently directed monetary policy--are fine with us, since such changes tend to produce steady trends. The problems arise when the changes are dramatic and sudden.
First, there are the situations where the upcoming volatility is foreseeable, such as with elections or the recent Maastricht vote. In such cases, our policy is to stay out of the markets during the period of greatest risk. To stay in a market where the direction of the expected volatility is unknown is not speculation; it's gambling. Our models simply do not apply.
The most difficult case is that of unforeseen future volatility--the Russian coup attempt, for example. Here there is no choice. There is an excellent chance you will be in the market when the news hits. The only way to deal with this eventuality is to trade small enough that such surprises will not be devastating. This we have learned to do.
Romaine: I would like to take issue, in a minor way, with your premise. We feel that the best systematic strategies evolve from strong concepts that can be tested using historical data. Systems based on such data are highly susceptible to form fitting. Developing trading models based on sound principles of mathematics and long-term macro economics makes them less vulnerable to unprecedented change.
Specifically, we use several techniques to reduce our exposure to the unexpected. First, we make extensive use of options in an effort to quantify and minimize risk. Second, we use multiple non-correlated systems so that no single event has a catastrophic impact on portfolio performance.
Q: Is rigid adherence to a trading plan necessary for success?
Dixon: If you read the Market Wizards books and talk with successful traders of all styles, you will find that one common trait among the survivors and winners is discipline. Adherence to a trading plan is an easy way to insure disciplined trading.
Eckhardt: General adherence to a good trading plan has been shown time and time again to be advantageous to the trader. The question is how rigid should adherence be.
Many who are now system traders began with a loose approach in which they had the experience of making mistake after mistake. This is often followed by a period of experimenting with systems, in which the trader finds that the market can trick him/her into making system changes at the wrong time, just like it used to induce the trader to make trades at the wrong time. Many come away from such experiences convinced that unquestioning adherence to a system is their only hope.
This approach has one overwhelming drawback. There is no way to improve; one gets stuck with the first viable system one develops. In an increasingly competitive environment, the choice is to improve or perish. I doubt that rigidity has much prospect for long-term survival.
Ostgaard: At Sjo we believe that a trading plan should represent codified common sense, experience, research, and as an absolute last resort, optimization. Of these, common sense is the most important, since it can transcend the limited time horizons of our experience and historical research.
Rigid adherence to a trading plan is not an absolute necessity for success as long as what overrides the plan is common sense. If it's something else--speculative scenario building, intuition, or second-guessing--then there can be trouble. The key is to know the difference. In general, a carefully thought out trading plan should very seldom need to be overridden, but if the necessity arises, it should be done.
Romaine: In short, a system that is not applied systematically is not a system. The value of a disciplined trading approach is that it allows you to design your strategy during nonstressful times. Then, when the markets are tough, you need only to execute your plan rather than being forced to face difficult decisions under pressure when you are most likely to make mistakes.
That traders who adhere rigidly to their systems are more successful is unclear. What is sure, however, is that they live longer.
Q: Mechanical trading systems, like mechanical machines, may need periodic maintenance. What type of fine-tuning do you advocate?
Dixon: When designing a system, I believe it's important to construct a set of rules which fit more like a mitten than like a glove. On the one hand, markets move in trends, but on the other hand, past results are not necessarily indicative of future performance. If you design a set of rules which fit the curve of your test data too perfectly, you run an enormous risk that it will fizzle under different future conditions. I believe the mitten-type approach will carry you farther into the future than the glove-type. Nonetheless, markets change and technology is relentless. Therefore, traders and systems must evolve.
Eckhardt: The question of fine-tuning system parameters merits a lengthier treatment than I'm able to give here. The short answer is that judicious fine-tuning can be helpful; most often such adjustments are detrimental.
A trading system can be thought of as a function from certain data configurations to certain expected outcomes. For a variety of reasons, this function ought to be continuous. This means that small changes in trading procedure should issue in small changes in expectation. If nudging a system parameter makes a large improvement in performance statistics, instead of looking to incorporate this tiny change into your practice, you should try to find out what's wrong with your data or your computations.
If your data set is large enough and your statistical procedures are otherwise unobjectionable, then small fine-tuning changes result in small changes in expected performance. The problem is that the latter tend not to be statistically significant. Now if you make a lot of small statistically insignificant changes, they might chance to meld into a significant difference; much more likely that the better performance statistics so obtained owe to historical over-fitting.
It is much more important to verify that your system is in a well-behaved region of its parameter space or to develop new macro-concepts, than to squeeze the last few drops of theoretical expectation out of your system through micro-adjustments.
Ostgaard: Fine-tuning smacks of optimization, so we do little of it. Rather, we are continually seeking new insights, new ways of looking at the markets. We try to increase our understanding of the way the markets work, test new ideas, and, if warranted, include these ideas in our system. In our view, conceptualization is the best of what research should be, and optimization is the worst. For us, one truly new idea is worth thousands of hours of computer time.
Romaine: In an area like futures trading, which is still maturing and changing rapidly, the need to upgrade and improve systems on an ongoing basis is great.
At RXR we look most intensely at structural changes, such as the evolution of new markets and shifts in liquidity in existing contracts. We also monitor certain performance characteristics of our strategies to be sure that the basic concepts which underlie them are still valid.
Q: Under what circumstances would you choose NOT to follow your system?
Dixon: Money management and risk control issues are generally my excuse for overriding my rules.
Eckhardt: In addition to unprecedented world situations, there are times when a system underestimates portfolio risk, often owing to uncustomary correlation among markets. In such situations you may have to override. However, if you find yourself overriding more often than say, twice a year, it is time to investigate whether the perceived defect in the system is real and whether it is repairable.
Ostgaard: Our system has been designed to operate in periods of normal interplay between supply and demand. We would choose not to follow our trading system at such times when external events are so extraordinary that we feel such interplay has been, or is about to be, abnormally disrupted in an unpredictable direction. However, an environment of gradually unfolding economic events represents our best trading venue. Economic "shocks" are our anathema.
Romaine: As I said before, the time to make changes in your system is not during the trading phase, but in the research process. Our traders have authority to determine the most efficient way to implement a signal, but not to override it. For example, if the model dictates that a put should be bought, we might choose to create a synthetic option using futures if the put is not priced at "fair value".
Q: What would you consider to be the most serious mistake made by novice system developers?
Dixon: Contemporary databases, software, and hardware allow system developers to test thousands of ideas almost instantaneously. I caution these people about the perils of curve fitting. I urge them to remember that one of their primary goals is to achieve discipline which will enable them to earn profits. With so many great tools it's easy to change or modify a system and to develop indicators rather than rules, but is it always wise?
I suggest they plan their research and conduct their studies in a systematic fashion. I remind them that systems and trading involve compromise--perfection is not achievable. Finally, I suggest they move slowly, that they look at trades on charts and not simply study printouts of P&Ls.
Eckhardt: There are so many fatal errors and so few ways to go right, that it's difficult to single out a most serious error. But I suppose for the system designer and trader, the cardinal sin is credulity--insufficient skepticism about your own ideas and results. Unwillingness to massacre, through rigorous testing, your own little creations. Needless to say, nothing you read or buy should be counted as more than a hypothesis to test and challenge.
Ostgaard: Most system developers lack a coherently developed market theory. Their "method" seems to be a disjointed effort to bring together a numerically and conceptually optimized set of parameters which work over a specified period of time. The result is often a better description of the noise in a time series than its meaningful trends and turning points. Further, there is a lot more illogic and outright superstition in system building than in most 20th century pursuits. System building should be scientific, not esoteric.
At Sjo we believe that for a signal to be meaningful, it should be eminently understandable. We should be able to explain--preferably a priori--why something works in a few simple, logical sentences. A good signal will almost always fit in well with our existing market model. The best signals--and the best systems--we have found, are intellectually as well as remuneratively pleasing.
Romaine: The number one error in developing a system is form fitting. Anyone can determine what system would have worked in the past. The key to system design is duplicability--that is, being able to generate the same kinds of return in the future that you have in the past.
In order to improve your chances, create your own system over one period, say 1983-1987, and blind test it over another, like 1988-1992. This will reduce the tendency to form-fit.
Copyright 2000 Barclay Trading Group, LTD.