BLACK BOX MODULE of the MARKET TRADER

Lesson 5: From Black Box to Neural Net

Learning objectives

1. The idea of non linearity

2. Neural Net Math

3. Neural Net Inputs

4. Neural Net Outputs

5. Orb Optimization in Neural Net

6. Options in Neural Net

General Notes

From the very beginning, I thought about this lesson as the final on Black Box module. The lessons 1 through 4 were devoted to different issues related to this module. The final lesson was intended as answers to your questions. But - while the lessons were released, questions were asked on the website and answered as soon as they were received ( Chat room for Black Box lessons ). Some of these questions became a reason for the latest patches of the Market Trader. It was a good example of our cooperation. 

As the result, the idea of the final lesson changed. Instead of answering your questions (don't worry, I will continue to do it on the website), I decided to discuss Black Box module in comparison to Neural Net - another powerful module of the Market Trader. It will be a bridge to the next series of the Market Trader's lessons - lessons on Neural Net. These lessons are supposed to be released in the end of February, 2004 (it will be announced). Also, this lesson 5 will introduce you to the idea of Neural Net.

1. Non Linearity as the Main Feature of Neural Net

There is nothing magic or fantastic in Neural Net. Neural Net technology is a special division of the modern mathematics that makes attempts to model processes occurring in a human brain. It is how Neural Net idea has begun - from the attempt to understand how the human consciousness work, how we (humans) make our decisions, how we think. When we deal with simple problems (like mechanics, for example), the simple logic of cause and effect works. It was the way of the science regarding to "dead" objects. We could measure such objects, find the relations between their speed and the distance, between their mass and the force (energy, power, etc) and their effect on other similar objects. More complicated systems have more complicated relations inside, and the connection between cause and effect becomes more sophisticated. When we deal with live objects, the complicatedness grows. Trying to understand and describe these new cause and effect relationships, mathematicians invented Neural Net technology. How is it connected to the tasks which might be solved the Market Trader?

Let me demonstrate one example. It is the case when Black Box model does not help.

 Suppose we are studying some process. We have found that it depends on the aspect between transit Mercury and the Sun. It means that some changes occur only at the moments when Mercury conjuncts the Sun within 5 degrees orb, and nothing happens at other moments. In other words, this is some hypothetical stock that depends on  Mercury conjuncting the Sun only. Here is how it looks:

This is a very good example for Black Box module, and you can do as it was discussed in the previous lessons: create the Black Box model for this stock and optimize it. There should be no problem, because this stock reacts according to the aspect between Mercury and the Sun.

Suppose that we continue to study this stock, and we have found another important factor: this stock is affected by the same Mercury - Sun aspect, BUT for increasing Moon this aspect (Mercury conjuncts the Sun) moves the stock in a positive direction and moves it negatively while the Moon is decreasing. In other words, the influence of the Mercury-Sun aspect depends on Moon's phase.

Now it looks like this:

Mathematically, the first example is the linear influence, i.e., the Mercury-Sun aspect is the only one affecting the stock and it does not depend on other factors. In this case, analyzing the stock, we can concentrate on this aspect only and forget about any other factors or any relations between them.

The second example illustrates the non linear influence, i.e., to estimate this aspect's influence, we should take into account other factors (like Moon phases), because these factors change the performance of this aspect drastically. In other words, the non linear relationship reflects more complicated connection between analyzed phenomena. We deal here not with the influence of separate factors - we analyze the whole system of factors which are related to each other. 

I think, everybody is familiar with the situation when you make a forecast for some market, and everything looks fine, but in real life this forecast has happened in an opposite way. We can call it as inversions (things have happened at the predicted time, but in the opposite direction). It looks like something has happened in the world that break the old laws. ...

If we try to make a forecast for the second example using the Black Box model, we find out that such a task is unsolvable (we can create the model, but the results of its application are bad). But the Neural Net resolves this task ideally.  

Why it is so? Let us create the forecast model for the second example using Black Box and Neural Net modules and compare them. 

I created the *.hyp file with three phenomena: a) transit Mercury conjuncts the transit Sun (orb=5 degrees), b) transit Moon is increasing in light, c) transit Moon is decreasing in light..

Here are the results of Black Box optimization for this model:

During the optimization process, the prediction curve (a thin one) bounces all the time which means that the Black Box cannot  find the good model. The program has got a right thing that on increasing Moon we have the upward move and on decreasing Moon it is downward. But the Black Box does not see the aspect, because the Black Box cannot make a conclusion how the Mercury - Sun  conjunction moves the stock (half of all conjunctions moves it up, while another half moves it down). The Black Box cannot see that the effect of the Mercury Sun  conjunction depends on the Moon's phases. It analyses each phenomena separately. So finally we have two weeks jumps only. The Black Box also cannot create a model for these two factors together - because the effect of the inner relationship between them is changing. When we discussed complicated Black Box models, we were capable to create such models only because each and every factor of them had the same effect on the analyzed market, and there were no connections between the factors (i.e., all factors were independent). 

Now let us solve this task with the Neural Net Expert. I will show the result of optimization after every 1000 steps:

Step 1000:

Here the Neural Net "understands" the influence of the Moon's phases only (two week jumps), it does not "understand" yet what is the aspect's impact on the stock.

Step 2000:

Now the aspect starts playing (the arrow points at such a place). But the Neural Net is not quite sure how this aspect works. The two week jumps are still strong here.

Step 3000:

Here the Neural Net understands that Mercury - Sun conjunction works together with the Moon's phases. Since that moment the Neural Net concentrates on combined influence. Now the same aspect works differently for increasing and decreasing Moon. The two week jumps become smaller. On the picture below, we can see the magnified piece of this diagram:

After 5000th step, we have a very good coincidence between our stock and Neural Net projection.

2. Neural Net Math: How It Works

Now let us talk some math here: how the Neural Net solved the above-mentioned example.

Neural Nets always have the same structure. It consists of three types of "neurons":

• inputs, or information we enter into the Neural Net. This information will be analyzed to find the "rules" applied to our problem; in our case, inputs are astrological events;
• outputs, or information we would like to get at the end; for stock markets, outputs mean the price forecast;
• one or several layers of hidden neurons. "Hidden neurons" are the core of the whole idea of the neural nets, they represent the complicated connection between inputs and outputs.

The central point of the whole thing is hidden neurons. Hidden neurons are a product of the neural net performance. The "good" neural net establishes the right connection between all its neurons - inputs, outputs, and hidden ones. For our example, 2 hidden neurons are enough. The optimized neural net looks like:

The idea is very simple. There are 3 inputs here : 1) Mercury conjuncts the Sun; 2) the Moon is increasing in light; 3) the Moon is decreasing in light. 

Two hidden neurons here are colored differently: one is red and the other is blue. The red neuron corresponds to the upward price movement; the blue one - to the downward. The neural net performance starts with the learning procedure; here the net gathers and analyzes information about its inputs and outputs (in other words, about astro events and the price). Some rules are applied for this particular Neural Net (we are not defining them previously, the Neural Net gets them itself during the learning process; it is what the Neural Net have learned):

Rule #1: The red hidden neuron is activated only when input and input are activated both, at the same time. This hidden neuron is connected to the output neuron (i.e., price) by the positive weight (the price goes up).

Rule #2: The blue hidden neuron is activated only when input and input are activated both, at the same time. This hidden neuron is connected  to the output neuron by the negative weight (the price goes down).

In other words, the Neural Net allows to set non-obvious connections between inputs and the price. 

3. Some Advice Regarding to Inputs for Neural Net

The Neural Net allows to solve many tasks easier and nicely. Let me show one more example. Suppose that we explore the effect of all major aspects between transit planets. We create this list of events (these are inputs for the Neural Net):

We have here 360 aspect pairs.

But we know that Back and Front aspects work differently. So we need actually to create two different lists. For Front aspects, choose "Front of" option and press "Add to List" button:

Then choose and press "Add to List" button once again. Now we have 720 planetary pairs.

Although, we know (from the past experience) that the aspect's influence depends on the type of motion of the planets involved. In other words, the aspect "direct Mars trine direct Uranus" moves the market in a different way than the aspect "retrograde Mars trine direct Uranus". We can consider this factor as well. In this case, push this button:

Now we have 2880 aspect pairs:

Using the Black Box, we could only define weights to each aspect from the list, compare combined influence of them all to the price in the learning interval, then define another weights to all aspects again, compare the new result to the price diagram, and do it again and again till getting the best fit - but still, we cannot consider such factors like back/front type of the aspect or direct/retrograde type of motion of the planets involved. .

What we get with the Neural Net in this case? We create 3 lists of events:

1) List of aspects - and it has no matter whether they are back or front or direct or retrograde planets form these aspects (360 events):

2) List for planets' location in respect to each other (like the Sun located back/front Jupiter) (45 events):

3) List for planets involved into the aspect in respect to their motion (direct/retrograde) (16 events):

In total, we have 421 events, and Neural Net allows to describe all possible situations (2880 combinations) using these 421 event. These events are input neurons.

How does the Neural Net work? Suppose that we know for a fact that the front aspect "direct Mars trine retrograde Uranus" moves the market very strong. For us, the most important thing here is the description of the aspect, not its impact on the market (at least, now). The Neural Net starts to learn. It must find this connection between the aspect and the market - because it is a fact. So, the Neural Net assigns the hidden neuron which has the only function - to be activated when the following four inputs are active:

1) Mars trine Uranus 2) Mars is direct 3) Uranus is retrograde 4) Mars is located in front of Uranus. This hidden neuron connects all these four factors together. Later, when the learning process is finished and Neural Net makes a forecast, every time when these four factors take place, the corresponding hidden neuron is activated and marks the price in some way..

Now, just imagine how many events we should take into account if the aspect's impact depends also on the planets' position in Zodiacal signs! Using the Black Box technology, we need to create events like "direct Mars in Aries trine retrograde Uranus in Leo, front aspect". Count yourselves all possible combinations.

But, using Neural Net technology, we need to add only 120 events (10 planets located in 12 signs) to 421 above mentioned events.

The big picture here is: using the Astrological Language, we record the words that Heavens use to speak to us. Then the Neural Net provides math that assembles these words to sentences that describe the analyzed phenomena (price movement, for example) the best possible way. From this point of view, the difference between Black Box and Neural Net is that the Black Box describes the World and Order using words only, while the Neural Net describes the same World using sentences.

Thus, here is a practical recommendation how to create inputs for Neural Net. The Neural Net type of thinking sounds like this:

"It looks like the transit aspects move the stock market, so I enter 360 transit/transit events to my model,

I read that back and front aspects work differently, so I add 45 events more for back/front aspects,

I know for sure (the fact observed by Sergey) that after ingressing Pluto in Sagittarius, aspects to Pluto start working another way. Ok, I add 120 events more - planets in Zodiac signs..."

The typical mistake of using Neural Net (and Black Box as well) is the usage of the huge number of inputs, hoping that the use of all possible factors gives better results. Instead, we face the so called overlearning effect when the Neural Net describes very well the happenings in the past (i.e., within the optimizing interval as you can always find enough words to describe everything you want), but forecasting results are bad (as there are so many words that the program cannot figure out what words are the most important ones).

4. Advice for Neural Net's Outputs

One more very important feature of the Neural Net is a possibility to use more than one output. In Black Box we can forecast just one thing: either price or turning points or  volatility. To forecast the other output, we must create a new Black Box model. With the Neural Net, we can analyze all these factors using the same Neural Net. Look at this example:

Here the same Neural Net does the projected model for Price (Relative Price Oscillator), for the moments when the stock has a big upward movement (more than 1%), and for turning top and bottom points as well. We look at the same stock from different points of view.

I highly recommend to use this method.

5. Orb Optimization in Neural Net

This Neural Net has been developed 3 years ago (in the beginning of 2001), together with Market Trader Millennium series. Since that, we came a long way and learn some new things. One of them is the orb optimization. Right now, the orb/shift optimization is the strongest part of the Black Box module. Sooner or later, this technology will be incorporated to Neural Net. However, this thing requires some additional studies, it is a rather complicated mathematical task. But the existing version of the Neural Net allows to do some orb optimization, too. We have there 2 Splite options:

The "Splite Culmination" works this way: the Neural Net explores the market's behavour around the culmination point. For example, if the value is 5 and inputs of the Neural Net are aspects, it means that the Neural Net deals with 11 variations for the aspect: the aspect's influence 5 days before its culmination, 4 days before, 3 days..., the aspect's influence at the culmination, then the next day after, 2 days, ..., 5 days after the culmination (do you remember Efficiency Test?). This option works only for events that have a culmination point (instanteneous events).

The similar thing is for the "Splite Aspect Orb". The only difference is that here we shift the orb's shape for a defined number of days. This option works only for events that occur within some orb (non-instant events).

6. Options in the Neural Net

There are several parameters in the Neural Net module you can vary. You will find them under "Learning" tab: 

Click on the "Options" button. You will get the window that allows to vary the default Neural Net options. The "Activation" panel represents features of the activation function between the layers of the Neural Net. 

The default parameters work well for 90% of all cases, you may  need to change the "Learning Rate" parameter only. Use these rules: Rule #1: if the projection curve changes too jumpy during the optimization procedure, you need to decrease the learning rate (for example, set it to 0.03). Rule #2: if the projection curve changes too slowly during the optimization procedure, you can increase this parameter. The momentum parameter allows you to avoid the "local minimum problem", keep it as it is. It makes the optimization process going more smoothly. 

I recommend to keep Activation parameters as they are. If you set all these options as "Linear", the Neural Net will work in the linear mode (Black Box model). So it is better to keep these parameters as they are. 

Also you can vary the "Sigmoid coefficient" parameter, it changes the projection curve a bit. But be careful while setting these options, you must be familiar with the peculiarities of the neural net technology to do it right.

You can also vary these options at  "Step 2: Neural Net" page:

Here you can define the type of the Neural Net - number of hidden layers. For most problems facing users of this program, 1 hidden layer is enough. 

I recommend to vary only one parameter "H1" (which is the amount of hidden neurons). You can  consider it as "complexity" or "degree of non linearity". My extended experience with this Neural Net has revealed only one type of the models when I have had to vary this parameter - "Price Bar Proportions" model. 

The following parameter is very important:

It points out the proportion between the amount of price points available for Neural Net optimization and the amount of events (inputs) used to create this model. 

I recommend to keep this parameter as 3 or more. If this parameter is lesser than 3, the Neural Net can fall into a memorize mode. Let say we have 100 price points and 100 astronomical events model; the proportion is 100/100=1. In this case, the Neural Net simply memorizes the price history: this astro event corresponds to this price; the next astro event corresponds to another price point, etc. This process is called over-training, or over-learning.  In one book, it is called "the effect of grandparents' care". We should have enough price points to avoid this effect.   

Another parameter to vary is "Redraw":

It indicates how often the program recalculates the projection line. In this example, it recalculates the projection line after every 1000 optimization steps.

The parameter allows to display the projection curve and price events in the same scale. You can play with it in one situation only: when you have a huge Neural Net (with a lot of inputs) and the projection curve looks almost as a straight line. It is because the optimization procedure takes a lot of time for big Neural Nets. In this case, disable this option. The projection line will be displayed in its own scale, so you can see the details.