Perturbation Modeling, Initial Approach
Last time I introduced some test data, and before that I formalized the Perturbation Model for Price Moves a bit further. Well this required me to rewrite the code I had written before for Sentiment analysis. I took advantage of interval trees to make my code fairly efficient, and also changed the way I initialize the price movements, yielding minor improvements over the naive methods.
The basic idea was that I created an object Perturbation, which has fields for influence, duration, and start time. Start time was a controversial decision for inclusion, but I ultimately decided it was best.
The Sentiment interface has been replaced by Modeler and contains one method:
public Perturbation[] getSentiment(Ticker ticker, int index);
As far as the initialization algorithm, I had previously initialized all sentiments to the price move observed in the interval of the news article. The biggest problem I had with this was that a corpus of just two news articles at exactly the same time would initialize to the price move observed in that time for both, when in reality, it should be half that.
So basically the new method works by splitting the data into elementary intervals by the distribution of news articles. For each elementary interval, the price move observed is distributed amongst all relevant perturbations, and then the resulting assignment is a weighted average of all of these observed price moves. I called this Elementary Average.
The code is below:
public Perturbation[] getSentiment(Ticker ticker, int index) { NewsCorpus corpus = ticker.getCorpus(); DataHistory data = ticker.getDataHistory(index); List<NewsStory> newsList = corpus.getNews(); Perturbation[] perturbations = new Perturbation[newsList.size()]; IntervalTree<NewsStory> intervalTree = new IntervalTree<NewsStory>(); SortedSet<Long> endpoints = new TreeSet<Long>(); for(NewsStory story : newsList) { perturbations[story.getId()] = new Perturbation(0,story.getTime(),forecast); intervalTree.addInterval(story.getTime(), story.getTime() + forecast, story); endpoints.add(story.getTime()); endpoints.add(story.getTime() + forecast); } Long last = null; for(Long next : endpoints) { if(last != null) { List<NewsStory> stories = intervalTree.get(last, next); double price0 = data.getData(last); double price1 = data.getData(next); double priceMove = price1 - price0; //System.out.println(priceMove); double distributed = priceMove / stories.size(); double length = next - last; double proportion = length / forecast / forecast; //System.out.println(stories.size()); for(NewsStory story : stories) { double current = perturbations[story.getId()].getInfluence(); perturbations[story.getId()].setInfluence(current + proportion * distributed); } } last = next; } return perturbations; }
Temporal interference works essentially the same as before, but now takes advantage of interval trees, making it significantly faster. Also, I added a parameter for learning rate. This allows large changes in the estimates initially, but slowly limits the amount that can change, which guarantees convergence, which was not necessarily guaranteed before.
The code is below:
public Perturbation[] getSentiment(Ticker ticker, int index) { NewsCorpus corpus = ticker.getCorpus(); DataHistory data = ticker.getDataHistory(index); List<NewsStory> newsList = corpus.getNews(); IntervalTree<NewsStory> intervalTree = new IntervalTree<NewsStory>(); Perturbation[] perturbations = initializer.getSentiment(ticker, index); for(NewsStory story : newsList) intervalTree.addInterval(story.getTime(), story.getTime() + forecast, story); double maxError = Double.MAX_VALUE; int iteration = 0; while(maxError > epsilon) { double rate = Math.exp(-iteration*learningRate); maxError = 0; for(NewsStory story : newsList) { double price0 = data.getData(story.getTime()); double price1 = data.getData(story.getTime() + forecast); double priceMove = price1 - price0; List<NewsStory> interferons = intervalTree.get(story.getTime(), story.getTime() + forecast); double sumInterference = 0; for(NewsStory interferon : interferons) { long intersection = forecast - Math.abs(interferon.getTime() - story.getTime()); double interference = perturbations[interferon.getId()].getInfluence() * intersection; sumInterference += interference; } double old = perturbations[story.getId()].getInfluence(); double error = priceMove - sumInterference; perturbations[story.getId()].setInfluence((old*forecast+rate*error)/forecast); maxError = Math.max(Math.abs(rate*error), maxError); } iteration++; } return perturbations; }
Of course, how well does this new method perform compared with the old method? For a baseline, I wrote a new NaivePriceMove Modeler, and analyzed the results for the 5 data sets. I used the correct duration for all of these tests.
Accuracy from Naive Price Move alone was: 0.637, 0.614, 0.566, 0.553, 0.568 respectively. Error was: 1.42, 10.83, 15.51, 6.32, 9.60!
Accuracy from Elementary Average alone was: 0.637, 0.613, 0.569, 0.553, 0.568 — almost identical. Error was: 0.312, 0.335, 0.336, 0.322, 0.328.
Note that though accuracy was almost identical, error (average squared) was significantly reduced in the case of Elementary Average–also more consistent.
Let’s see how temporal interference performs as initialized by either.
Temporal Interference initialized by Naive Price Move was: 0.944, 0.902, 0.887, 0.852, 0.889. Error was: 0.032, 0.065, 0.114, 0.097, 0.070.
Temporal Interference initialized by Elementary Average was: 0.943, 0.902, 0.887, 0.852, 0.889. Error was: 0.033, 0.065, 0.114, 0.098, 0.071.
As we can see, the results are almost identical. This was interesting to me, and made me wonder whether or not the initial values even matter. So I tried two more experiments, one in which I initialized the influences to 0. The other where I initialized to random values between -1 and 1.
In the case of zeroes, the accuracy was obviously 0, and the error floated around 0.33 pretty closely. When we initialize T.I. with zeroes, there is no change in accuracy or error at all.
In the case of random initialization, the accuracy was around 0.5, and the error floated around 0.66 pretty closely– as expected. When we initialize T.I. with random values, we did see some change. Accuracies exhibited were: 0.889, 0.864, 0.829, 0.799, 0.837. Errors were: 0.064, 0.108, 0.179, 0.156, 0.117.
So it doesn’t converge to some value inherent to the system regardless of what we initialize to, but it does depend on what we seed, however, only to some degree. I think the the zeroes case reduces to initializing with Naive Price move, but I may be wrong. Also, Naive Price Move and Elementary Average both tend to result in the same direction of price move, so maybe that’s important.
There is one more test to run, and that’s a throwback to whether concurrent news articles introduce any major problems. I generated another data set that creates two news articles at each time point. Because there is no way to tease apart which one is good and which one is bad here, we expect low performance. But we want to find out if one method performs better than the other on this particular test.
I used T.I. for all tests, and varied the initializer. Initialized with zeroes, we saw an accuracy of 0.709, error of 0.280. Initialized with random, we saw an accuracy of 0.609, error of 0.548. Initialized with N.P.M, accuracy 0.517, error 12.619. Initialized with E.A., accuracy 0.708, error 0.280.
Interesting! Initialing with N.P.M. appears, as expected, to have disastrous results when there are news articles at the same time. It actually performs worse than random initialization! Also, note that zero-initialization seems to reduce to E.A., rather than N.P.M. as I had initially anticipated.
Now there is one final question to this whole analysis, efficiency. Perhaps one method converges more quickly than another. I counted the number of iterations taken to solve data set 4, initialized by different methods. Zeroes: 388. Random: 379. N.P.M.: 385. E.A.: 388.
It appears that there were no major differences in runtime. Perhaps it is more strongly related to the learning rate than anything else. Again, we see zeroes equaling E.A., indicating that perhaps the best idea is to simply initialize with zeroes strictly for code simplicity.