Evolution of cyclical analysis in Timing Solution

Written by Sergey Tarassov

Introduction

In this article I want to share my personal view of the evolution of cyclical analysis applied to finance data.

In the early 1990s, I came across to the problem of a possibility to forecast financial data series. I worked then in the nuclear science field. Due to that, I had an access to scientific software libraries (like CERN library), special mathematical packages (like MatLab) and simply to the libraries specialized in scientific papers in physics and nuclear physics. The cyclical analysis was one of the main tools in these fields. Also, at that same time, the financial data series became available, too. The logical question would be: is the cyclical analysis of any use to the stock market? I have studied this question for some time. To understand it better, in the mid 1990s I have started  developing a software to forecast stock market moves.

Now, in 2022, I can say that the cyclical analysis presented in Timing Solution software is something different compared with what I knew at a start. This is another branch of the mathematics, and this difference will be explained.

First periodogram - first confusion

In the beginning (1990s), the problem looked quite simple. On one side, there were stock market players, with a very basic knowledge of math and some basics of statistics. And there were scientists - physicists, mathematicians, statisticians, with vast knowledge of many things cyclical. So, someone has to help the first ones to discover and apply the wisdom of the other ones. If there are any cycles at work in financial data, we can find them easily (and then make working market models).

I have started with my first periodogram for a set of financial data. Following the scientific style of research that I was used to, I took the longest available data set. It happened to be Dow Jones since 1885. Then I conducted a necessary preprocessing procedure (eliminating a trend). If there were some cycles at work there, I was expecting to see them right away. Instead, I got something like this:

It was my first confusion. A spectrum like this one usually indicates that this process is not a cyclical one, it is impossible to find certain cycles there. This periodogram looks pretty "white-noisy", the energy is distributed among all cycles there. For any scientist looking at a spectrum like this one, the further discussion of cyclical modeling of this process is pointless. As an example, look at the book by E.E.Peters, "Chaos and order in the capital markets". They have calculated spectrum, found that the spectrum is very noisy, and they stated that it is impossible to forecast stock market movement using spectrum.

Applying different algorithms to calculate spectrum does not change the picture at all. I have incorporated in Timing Solution software some variation of Fourier transform: not for the price, but for a covariation function. The idea is very simple. Suppose that 57-days cycle is present for some financial instrument data set. It means that if we calculate a covariation between initial chart and that same chart shifted 57 days (take it as a simplified correlation), we should get a high covariation value; the presence of 57 days cycle means that this pattern repeats itself approximately every 57 days. Hence high covariance values for some shift means that this shift coincides with a supposed working cycle for this financial instrument. In Timing Solution this option is presented here:

With that, here is an improved periodogram; it is pretty much the same as above:

Many years later, with the development of Q-Spectrum 2 module, all these academic beauties have been removed. I have found that this is not an important factor.

Idea! - temporary vs permanent cycles

The first good solution has been found when I started to apply multiframe spectrum. At that moment I studied wavelet analysis. It became clear to me that, instead of looking for permanent cycles, we should analyze temporary ones, the cycles that work for some restricted period of time. Cycles in finance do not live forever (except economical cycles), and we should take this into account.

This idea led to introducing a new parameter, SM- stock memory:

This parameter represents cycle's life expectancy. As an example, if we analyze 100 days cycle with SM=12, the life expectancy of this cycle is 12x100=1200 days. The spectrum calculated this way looks less noisy:

It indicates a presence of two cycles at least. However, we should remember that these cycles do not live forever. In a couple of hundred days, the cyclical "portrait" will be changed. We deal now with temporary cycles.

Welcome to Walk Forward Analysis (WFA)

Switching the forecast focus to temporary cycles was a big step. Still, there were many questions not answered and not understood yet. The year 2015 came. At that time, I got a clear understanding that there are two absolutely different cyclical analysis: 1) cyclical analysis that is based on curve fitting criteria - we use it when we need to explain something in the PAST; and  2) cyclical analysis that is oriented to make a forecast - we are more oriented to forecast the FUTURE.

Actually, it was a big surprise to me. I did not expect that this shift from a curve fitting to forecasting brings big changes in the program's development and its application. It was against my previous scientific experience: building a model that explains rather well the system's behavior in the past was a sign that we automatically get a key to its future behavior. It does not work this way in finance.

That is how Q-Spectrum module was born. In it, we totally rebuild Fourier transform procedure based on results of WFA (walk forward analysis). Originally WFA has been developed by Robert Pardo to verify the efficiency of a trading system. We have developed our own WFA protocol to verify projection lines, it is explained here: http://www.timingsolution.com/Doc/level_2/WFA/index.htm

Immediately I have got confirmation from the software users that this new, customized, cyclical analysis is better for forecasting the market moves. From the other side, it was a first step entering an absolutely new territory, with new mathematical entities; so far I have not the final answer as to how work better with these entities.

Just an example: 1) we faced with INVERTED cycle, long time I rejected these cycles taking them as a kind of artifact. Now we can see that this is not an artifact, it is a new reality. 2) classical decomposition methods (building projection line based on cycles) do not work for inverted cycles. We are trying several alternative variations, and I am certain that some theory regarding this subject should be suggested.

Q-Spectrum looks this way:

There are positive and inverted cycles there. Plus, as you see, more energy is stored in inverted cycles than in normal non-inverted cycles.

There is also one extra problem here: to get some results, an extensive back testing needs to be conducted; it is a very time consuming procedure, even with modern computers. The good old times when inventions and theories were made with the help of pen and paper are over.

27 October, 2022

Toronto, Ontario

Canada.