Backfill using IB as default

[soulfire]

Hi,

 

I was wondering if there are any noted current issues with backfill. Normally backfill is lightning fast updating all symbols. When I tried using it Friday and today, it's now acting very slow/sluggish taking minutes to process where prior it took seconds. Stocks process slow but completed. I had to initiate backfill on the chart directly in some cases for futures data. My data sources are IB, Ameritade, Tradeking, and Yahoo finance. This is the first time I'm experiencing such a slowdown.

 

Update: This is the error message that was triggered- not sure if it's related:

***********
10/08/16 8:24:27 PM: MT.Util.MTErrorObj:  Type=Error Severity=Medium Action=Log, ShowIndicator Code=OldOffsets Length not equal new offsets length 60 119 Message=
Details=AppVer: 1.1.1700.2018 Err.Flags: Log, ShowIndicator

 

 

Edited October 8, 2016 by soulfire

[Jerry Medved]

how much data do you have for the symbols in question?

[soulfire]

Data goes back to April of this year for some. For others, just a few weeks, such as @ymz16,@nqz16,@ymh17, etc...

@esz16 loaded after a while, as did @esh17, but others didn't on the first pass and needed multiple or direct chart attempts.

[Jerry Medved]

with that much data, it may take much longer than normal to merge the incoming BF data with the data that is already there, especially for high volume symbols that would have millions of rows.

A quick test would be to open a chart for a dummy symbol - lets say TEST. Then do a "Backfill As..." and specify @ESZ16 as the symbol. That way MT does the @ESZ16 backfill but puts it into TEST chart. Is that fast or also slow?

[soulfire]

Data seems to load at equal speed when applied directly to a backfill activated on the chart. @ymz16 loaded at the same speed using dummy test symbol chart and actual symbol chart.

The delay seems to stem from using "backfill all" on the portfolio window. I do portfolio backfills weekly, and up to this week, the backfill have been so fast, I had to double check to see if they actually completed, which they did. This is the first time I'm seeing it take so long to go through the process, and some symbols aren't getting updated.

[soulfire]

I forgot to mention this anomaly that also happened Friday but will now include in case it's related- when I started up the program yesterday, I got the "using free service symbol/portfolio limitations" warning even though I had my Trade King account verified weeks earlier and was operating without issue. I restarted, and this time the program started up without a problem.

 

 

[Jerry Medved]

ah, ok the backfill all with IB is a different issue.  IB has throttling so you re restricted to like 60 requests per 10 min.  MT slows down the requests to try to avoid that but if you are doing Backfill all on a large portfolio, you will end up blowing through the limit and then the rest of the backfills will fail.

As for free service notice and TradeKing - it is not like your account gets flagged once and that is it.  MT assumes you will actually be using TK. If TK is not used in MT for a while, your MT account subscription will expire.

[soulfire]

I've never had any such slowdown in the past- this week is the first occurrence of such behavior with no change in my portfolio size.

I have two futures porfolio's of about 24 symbols each, and one stocks portfolio of 45 symbols. I click backfill on only one portfolio at a time which seems to update only that portfolio rather than all. One a portfolio is done updating, I select backfill on the next. The slow down is happening with every portfolio, starting with the first one selected.

" MT assumes you will actually be using TK. If TK is not used in MT for a while, your MT account subscription will expire. "

I'm confused. The "Tradeking Promo" made no mention of being required to place trades through MT, or even on Tradeking. The only requirement was funding a Tradeking account and adhering to their terms. I also only opened my TK account a few weeks ago, so even if the account had a trading requirement - that would mean one is expected to place trades at least monthly or sooner?

 

[Jerry Medved]

IB - don't know. If you can recreate, do so then send the log:
go to FILE / Export/Backup menu (from Dashboard or Portfolio)
Select “To share with others” export file protection option on top right
Leave the checkboxes already checked, and also check LOG FILE.
Export, then email me the exported file please.

TK - you don't have to trade, but there is a minimum balance requirement as well as just a periodic check that you are using TK in MT. When MT checks, it updates the MT account accordingly to be enabled for a while. So if the account is used in MT (quotes, etc) at least every few weeks, you will not have an issue.  for TK MBT accounts, the login has to be via the Trading portion (so account view at the minimum, not just quotes)

[soulfire]

Thanks- if the slowness persists, I will send you the log files.

I have my TK account set as a data source for quotes and as the trading interface. Does TK need to be set as the default data source?

[Jerry Medved]

MT doesn't care if it is default or configured. It only checks when the login actually occurs. If you view the account info for example for the TK account, that will force the login. if the balance is $500 or higher, that should do it.