Backtesting?

[zenlando]

I've been looking into different backtesting apps recently, and it occurred to me that MT is possibly already well on its way to getting some kind of backtesting feature implemented if the choice were made to do so. The reason I say that is that paintbars are done, and essentially all the indicators have now been implemented. And it seems that writing and saving paintbar rules creates various logic operations using specific indicators, including the "and" "or" as necessary, and might form a good foundation for some kind of simple backtesting app for ETFs and Equities? Option backtesting would be amazing, but even a start with just being able to backtest ETFs with the ability to control historical start and stop times would be pretty awesome.

But of course I'm not a coder, so I have NO idea how much time it would take to actually tie what's already there now in MT to historical data of varying chart frequencies and the like to make something viable . But that said, it seems that most of the more time-consuming features created for MT have now been implemented. Food for thought... would really be handy to have as part of MT at some point.

Thanks, Dave

[Mike Medved]

It depends on what you want in your backtesting...

As you said, for some definitions of backtesting, MT is basically there. All you have to do is implement your backtesting strategy as a paintbar on historical or intraday scale, and run it on a particular chart on as much data as you can get in backfilling from your particular source. On any data - including options.

Once you're satisfied in your strategy's viability, just leave the paintbar running on that chart and it will be able to notify you when to buy/sell in real time

But if you want backtesting to be run on the whole universe of stocks, that would be problematic. So would scanning be. Mostly because we're not in the business of providing market data to users. We leave it to other people who have the means to do it and do it well (for some values of "well").

[zenlando]

Thanks Mike, I can sort of see what you're saying, but if I thought I could do the kind of backtesting I'm envisioning right now on MT I wouldn't be looking for apps . What I think I'm getting at is this: Let's say I wan't to backtest a simple EMA crossover on a 65 minute chart, and I want to see what the P&L is for trading SPY for a specific three month period that I knew was a market downturn in 2015. ideally I would just need to plug in the OHLC at 65 minute somehow, then select the two different EMAs I want to cross, then somehow click on "run", and it would then show all instances of the crossover for that three month period, along with a P&L for each instance, along with a P&L total for the chosen time period. If it showed all these points on a chart, great, but in a way not necessary.

Of course it would need to be able to be set up for any time period consistent with appropriate historical data availability in MT, as well as have its own simple UI that's made for this backtesting purpose. My point in the earlier email was that it seems it's possible that much of the development work has already been done with the implementation of all the indicators and the ability to create rules for paintbars. If I'm missing something here, please let me know...

[Mike Medved]

What you're suggesting, the paintbar can already do (well, with a bit of effort in coding it in the paintbar itself).  You can simulate trades, calc P&L and output it as an alert etc, all in the code.

The problem would be the data. Paintbars run on charts, charts do not have more than 120 days of intraday data.

 

 

[Doug Hayman]

Mike (or Jerry),

I have an old Wealthlab system (Pascal-based) that I still use for back-testing, but unfortunately it doesn't support Range Bars, which has become a favorite in my trading.  I'm looking into trying to accomplish this in MT, if possible.

Paintbars are great for real-time trade alerts, but if I want to test my logic over 120 days, it is cumbersome to have to manually inspect trades, profit, etc.

I am intrigued by your post above, where you suggest that Paintbar logic can simulate trades and alerts.  As far as I can see, Alerts in Paintbars can only add visual or audio alerts, but cannot record/simulate trades.

1) Could you provide any examples on how I could send simulated trades (using a simulated acct) to the Alert Mgr, which in turn, could be fed into the Account View screen, which could then automate the calculation of P/L?

2) As an aside, are there any plans for any sort of crude back-testing facility being released in MT, or is your focus in developing a trading API that can be accessed in Paintbar logic?

Thanks in advance for any advice you can provide on the above.

[Doug Hayman]

12 hours ago, Doug Hayman said:

One additional question please:

3) I'm not a C++ guy, but is there a way from within Paintbars to write out to a text file (I'm thinking .CSV) via either native compiler C++ procedures or any linked libraries that you use?

 

[Mike Medved]

We are working on a streaming API that will be used to connect external programs to MT to get streaming market data (L1 and L2) using MT sources, get accumulated market data, get alerts as they are triggered, add/remove/modify system alerts, monitor trading accounts and place/cancel/modify trades. Right now everything other than that last trades part is done.

We're thinking about how to do backtesting.

Writing to files from paintbars is disabled for security reasons.

[Doug Hayman]

1 hour ago, Mike Medved said:

OK Mike, cool!  Using Advanced State logic, I could implement a poor-man's back-testing system now for a given intraday chart symbol, including keeping track of W/L, profit, and even day, week, month roll-up.  The current problem is that I can't write those results out anywhere, including as an Alert.  Looking forward to that API!

 

[Mike Medved]

You can write things out in LogString()  - and then you can look at what you wrote using https://docs.microsoft.com/en-us/sysinternals/downloads/debugview

 

[Doug Hayman]

34 minutes ago, Mike Medved said:

You can write things out in LogString()  - and then you can look at what you wrote using https://docs.microsoft.com/en-us/sysinternals/downloads/debugview

 

Oh, interesting. I think you just resurrected my interest in a current backtesting system. Thanks for the heads up.

[Mike Medved]

There is other logging going on, so filter it on "PAINTBAR" so it catches only your log strings.

[Doug Hayman]

4 minutes ago, Mike Medved said:

There is other logging going on, so filter it on "PAINTBAR" so it catches only your log strings.

Yeah, I just downloaded the .EXE and tested this.  This will definitely work for me.  Need to think about it some, but I think I'll generate a concatenated CSV record for each trade in my Paintbar, filter output from Debug log file, and then parse/import into Excel, to do post-analysis, roll-ups, etc.   Thanks again for sharing.

[Doug Hayman]

OVERVIEW:

This is a first cut at a crude MT-based back-testing system I came up with using Mike's idea (above) of using the LogString() function and the Microsoft tool DBGVIEW.EXE
to capture Paintbar output.

QUESTION FOR MEDVED TEAM:

  Not sure what the policy is for uploading files on this forum, but which of these can I upload and attach to this post?

1. Parse.Bat (source code displayed below)
2. ParseToCSV.exe (source code display below)
3. BackTest.xlsx (spreadsheet to load and post-process back-testing CSV data).  Created using the lowest common denominator version I could find - Excel 2010, and contains no macros.  This is a key element for displaying backtest results.

   NOTE: If I'm unable to attach these in forum, anyone can Email me to send these.

LIMITATIONS:

1. Can only test 1 intraday symbol chart at once;  although I didn't test it, this system should work on historical data as well;
2. Only 1 trade for symbol (long or short) can be active at once;
3. # of Shares/Contracts being tested (all or nothing) should be assigned in variable "NumSharesContracts" in code;
4. Commission per Share/Contract should be estimated and assigned to variable "Commission". For fixed commissions, divide the commission by # Shares traded;
5. Enter the value for ShareContractValue variable.  Should be '1.00' for stocks, or point value for contracts;
6. All dates/times of generated CSV records are in UTC, so if you want to synch up your trades with your Intraday chart, you will have to do the necessary date math (for example, as of this posting, UTC times are 4 hours ahead of EST times).  Of course, you can change your chart's timeframe to UTC too;
7. Best to run this backtesting facility when market isn't active; otherwise you may get repetitive signals for current Bar;

INITIAL ONE-TIME SETUP:

1. Create a Back-testing folder on your computer;
2. Download DBGVIEW.EXE from Microsoft website to this folder (see Mike's post above for link);
3. Create a DOS .CMD or .BAT file (I call it Parse.BAT) in folder, using Code for BAT file below.  NOTE:  I couldn't figure out how to Parse through a file in DOS and grab data from a certain line position, so I wrote a small VB program (ParseToCSV.exe) to accomplish this, which is called from the .BAT file. The source code is below, which can be compiled on your end, or you can replace the ParseToCSV.exe line in the .BAT file with your own parser.

CODE FOR PARSE.BAT:

  =======================================================================
  @echo off
  rem The following DEL command will generate error if run for first time
  del/q dbgview.csv
  findstr " PAINTBAR:" dbgview.log > dbgview.txt
  ParseToCsv.exe
  =======================================================================

CODE FOR PARSETOCSV.BAS (used to generate ParseToCSV.exe, called in PARSE.BAT):

  The generated .EXE was compiled using VB6.0 32-bit, which should run on all
  MT machines:
  ===================================================  
  Attribute VB_Name = "ParseToCSV"
  Option Explicit

  Sub Main()
            
  Dim InputFile, OutputFile, Line, NewLine As String
  Dim Length, PaintbarPosition As Integer

  InputFile = "dbgview.txt"
  OutputFile = "dbgview.csv"

  Open InputFile For Input As #1
  Open OutputFile For Output As #2

  Do While Not EOF(1)
     Line Input #1, Line
     Length = Len(Line)
     PaintbarPosition = InStr(1, Line, "PAINTBAR")
     NewLine = Mid$(Line, PaintbarPosition + 25, Length - (PaintbarPosition + 25))
     Print #2, NewLine
  Loop

  Close #1
  Close #2

  End Sub
  ===================================================  

ADVANCED PAINTBAR CODE FOR BACK-TESTING (need to create an Advanced Paintbar):

  1) You will need to add your Buy/Short criteria logic yourself;
  2) There are some paintbar/audio options in the code, that you can choose to add or not;
  3) You will need to add your Sell/Cover exit criteria logic yourself;
  4) You will need to add MT "State" Logic templates to your code (see the STATE
     tab in Advanced mode), and you will need to add my logic to the:
     a) PaintbarInitialize() (see below)
     b) PaintbarState structure (see below)

  Here's the full Back-testing Paintbar Code:

  private void PaintbarInitialize()
  {
      string TradeCSVHeader;
      TradeCSVHeader = "SYMBOL,START DATE/TIME,START ORDER TYPE,SHARES/CONTRACTS,START PRICE,END DATE/TIME,END ORDER TYPE,END PRICE,PROFIT";
      LogString(TradeCSVHeader); // writes to the DBGVIEW.EXE output window                               
  }
  /// <summary>
  /// Holds paintbar state - fill with variables needed to be preserved for next paintbar calc
  /// </summary>
  private struct PaintbarState
  {
      public bool   InTrade, LongTrade;
      public string StartOrderType, EndOrderType;
      public string StartDateTime;
      public double StartPrice;
  }

  public void MainCalculation()
  {   
  string TradeSymbol = "@MESU20";   // Enter symbol you are back-testing here
  int    NumSharesContracts = 1;    // Enter # of Shares/Contracts traded in back-test
  double Commission = 0.94;         // Round-Trip, per-share/contract commission
  double ShareContractValue = 5.00; // Value of each point. Enter 1.00 for stocks; point value for contracts
  double Profit;
  string EndDateTime;
  double EndPrice;
  string TradeCSVString;
 
  //  Condition for Long Trade
      if (ENTER YOUR LONG TRADE LOGIC HERE, ALONG WITH FOLLOWING CONDITION)
          & (!CurrentState.InTrade)
      {
           CurrentState.InTrade        = true;
           CurrentState.LongTrade      = true;
           CurrentState.StartOrderType = "Buy";
           CurrentState.EndOrderType   = "Sell";
           CurrentState.StartPrice     = Close; // This will be depend on your strategy
           CurrentState.StartDateTime  = Timestamp.ToString();
           SetColor(SysColor.MainIndicator2);  // OPTIONAL: paints chart for Buy if PB used in real-time
           TriggerAlert ("Buy Stop");  // OPTONAL: provides audio alert if PB used in real-time
       }   

  //  Condition for Short Trade    
      if (ENTER YOUR SHORT TRADE LOGIC HERE, ALONG WITH FOLLOWING CONDITION)
          & (!CurrentState.InTrade))
      {
           CurrentState.InTrade        = true;
           CurrentState.LongTrade      = false;
           CurrentState.StartOrderType = "Short";
           CurrentState.EndOrderType   = "Cover";
           CurrentState.StartDateTime  = Timestamp.ToString();
           CurrentState.StartPrice     = Close; // This will be depend on your strategy
           SetColor(SysColor.MainIndicator3); // OPTIONAL: paints chart for Short if PB used in real-time
           TriggerAlert ("Sell Stop"); // OPTIONAL: provides audio alert if PB used in real-time
      }
    
      //  If currently in a trade, Check to see if current trade should be exited    
    
      //  Check for Long position exit
      if ((CurrentState.InTrade) & (CurrentState.LongTrade))
      {
          if (ENTER YOU LONG TRADE EXIT LOGIC HERE)
          {
              CurrentState.InTrade = false;
              EndPrice             = Close; // This will be depend on your strategy
              EndDateTime          = Timestamp.ToString();
              Profit               = ((EndPrice - CurrentState.StartPrice) * NumSharesContracts * ShareContractValue) -
                                            (Commission * NumSharesContracts);
              TradeCSVString       = TradeSymbol + "," + CurrentState.StartDateTime + "," + CurrentState.StartOrderType + ","
                                     + NumSharesContracts.ToString() + "," + CurrentState.StartPrice.ToString() + "," + EndDateTime +
                                     "," + CurrentState.EndOrderType + "," + EndPrice.ToString() + "," +
                                     Profit.ToString();
              LogString(TradeCSVString); // writes to the DBGVIEW.EXE output window
              SetColor(SysColor.MainIndicator4);  // OPTIONAL: paints chart for Flatten Position if PB used in real-time
              TriggerAlert ("Flatten Position"); // OPTIONAL: provides audio alert to flatten position if PB used in real-time
          }
      }

      //  Check for Short position exit
      if ((CurrentState.InTrade) & (!CurrentState.LongTrade))
      {
          if (ENTER YOUR SHORT TRADE EXIT LOGIC HERE)
          {   
              CurrentState.InTrade = false;
              EndPrice             = Close; // This will be depend on your strategy
              EndDateTime          = Timestamp.ToString();
              Profit               = ((CurrentState.StartPrice - EndPrice) * NumSharesContracts * ShareContractValue) -
                                            (Commission * NumSharesContracts);
              TradeCSVString       = TradeSymbol + "," + CurrentState.StartDateTime + "," + CurrentState.StartOrderType + ","
                                     + NumSharesContracts.ToString() + "," + CurrentState.StartPrice.ToString() + "," + EndDateTime +
                                     "," + CurrentState.EndOrderType + "," + EndPrice.ToString() + "," +
                                     Profit.ToString();
              LogString(TradeCSVString); // writes to the DBGVIEW.EXE output window
              SetColor(SysColor.MainIndicator4);  // OPTIONAL: paints chart for Flatten Position if PB used in real-time
              TriggerAlert ("Flatten Position"); // OPTIONAL: provides audio alert to flatten position if PB used in real-time
          }
      }
  }
  =============================================================================

SAMPLE .CSV OUTPUT FILE GENERATED

  Here's a small excerpt of a .CSV file created from one of my Intraday strategies, for @MESU20:

  SYMBOL,START DATE/TIME,START ORDER TYPE,SHARES/CONTRACTS,START PRICE,END DATE/TIME,END ORDER TYPE,END PRICE,PROFIT
  @MESU20,6/2/2020 1:31:07 PM,Short,1,3046.25,6/2/2020 1:37:37 PM,Cover,3045.5,2.81
  @MESU20,6/12/2020 7:12:22 AM,Buy,1,3040.25,6/12/2020 7:19:22 AM,Sell,3041.75,6.56

TO EXECUTE BACK-TESTING SYSTEM END-TO-END:      

1. Invoke DBGVIEW.EXE; if it's already running, kill the process and re-invoke.
2. Execute Back-Testing Paintbar Code.  You will have to reload your layout containing the Back-Testing Intraday Paintbar. This will generate trade log file entries in the DBGVIEW.EXE window;
3. Save the output generated in the DBGVIEW.EXE windows to your back-testing folder and name file DBGVIEW.LOG;
4. From a DOS session, Run Parse.bat.  It will generate DBGVIEW.CSV, which are a sequence of CSV records that include all information associated with all simulated trades that met your criteria (one line per trade);
5. Load DBGVIEW.CSV into either my spreadsheet (the "dbgview" sheet of BackTest.xlsx) or into your own spreadsheet, if I'm not allowed to upload it, for analysis. In my spreadsheet, load/overwrite contents of the "dbgview" sheet in BackTest.xlsx with DBGVIEW.CSV.  

            WARNING: DO NOT "DELETE" contents of existing "dbgview" sheet to accomplish this; alternatively,
                                use "CLEAR ALL".  If you "DELETE", the formulas in the "TradeResults" analysis
                               sheet will result in #REF! cell references.
            Back-testing results for this loaded data set will appear in summary in the "TradeResults"
            sheet. Also, you may want to use the Format->Autoformat feature to make the "dbgview"
            sheet more readable.  The TradeResults sheet can be enhanced to add additional back-testing
            summary data (for example, daily or weekly roll-up).

      6.  Again, you will have to kill the DBGVIEW.EXE process and re-start it to re-run or execute a different Back-test (yes, this is very crude), since an active DBGVIEW.EXE
           session will have accumulated data in it since its invocation.

 

[Jerry Medved]

send the zip with the files to support. I will see about getting it posted

[hurdleda]

Any news on backtesting feature for MedVed

[Jerry Medved]

nothing new.