Memory Utilization on Latest Beta

[Doug Hayman]

I always have a lot of stuff running on my 16GB (memory), i7-4core-3.6Ghz computer, but today I noticed that I had extremely high MT memory utilization (3.1 GB).  Only 20 tickers in my Portfolio, 2 intraday charts, 1 historical, 1 news, 1 ToS, 1 DOM, and several Account view screen.

I noticed that refresh on my Intradays was really slow when switching between ticker symbols.   That's when I checked and noticed the very high Memory Utilization.   Never had any MT application lag like this before.

I restarted MT, and was back down to about 1.2GB of memory usage on the same MT screen.   Still noticing the lag though when switching between symbols.  And memory utilization climbing when I switch between tickers.

The big change I've made since first noticing this problem, is the addition of the 2nd Intraday chart.  Got rid of this 2nd Intraday though, and lag still exists when switching between tickers, and Memory utilization continues to grow.

If I stay on the same ticker symbol for awhile, memory footprint decreases eventually to below 1GB.

CPU Utilization is always fine.  I sent you logs.

Edited August 6, 2021 by Doug Hayman

[Jerry Medved]

you have a paintbar that is doing something very odd

you are doing logString in the paintbar which is slowing things down drastically

[Mike Medved]

LogString is there for debugging. Once you debugged, take it out

[Doug Hayman]

Dang, good catch.  Forgot to comment out; I use that for back-testing stuff.

A little bit snappier, but still sluggish, and memory footprint grows significantly, when toggling between symbols.

[Jerry Medved]

do you use 120 days of intraday data? If not, then change the settings on SETTINGS / CHARTS / GENERAL. That will have a pretty big impact on charts. # of days of tick data too - if you don't need 5, lower it. 

[Doug Hayman]

Yeah, I use 120 days of intraday for backtesting purposes, whenever I backtest (today, for example), so I don't want to prune that.  I've been playing with Tick data, but I can lower that value.

Still, I don't recall the sluggishness in Intraday charts, when switching ticker symbols.  Specifically, the 2nd Intraday chart that I just added, seems to lag the rest (maybe it's updated last, so has that effect).   Lag hardly noticeable much when switching between stock symbols in my portfolio; much more noticeable when switching between future symbols @MESU21 and @MNQU21, and screen refresh delay is most prominent on my 2nd Intraday chart.

EDIT:  Will deleting symbol from portfolio, and then re-adding, effectively prune the data for that ticker symbol?

Edited August 6, 2021 by Doug Hayman

[Doug Hayman]

I lowered my tick data to lowest value (1), and that has not made a difference.

Is there any suggestion that you might have that would allow me to "archive" intraday data for above symbols periodically (for later back testing retrieval), so that I could then lower the # of intraday days for charts?

I'm consistently using 3.5GB - 4.0GB+ of memory  when I switch between symbols (which currently have about 90 days of intraday data currently).  If I stick with a specific ticker symbol for a period of time (that is, no switching between symbols), the memory footprint goes down after awhile (I'm guessing due to garbage collection).

Thanks in advance.

 

[Jerry Medved]

you restart MT since making the change?

[Doug Hayman]

Yes, I restart every day.  I should have stated above, when I stay with same symbol, memory footprint reduces to below 1GB.

I also removed all "hidden" indicators from my charts, so I'm pretty much running bare bones, and still high mem utilization.

[Jerry Medved]

I need to get the log file and data to check. Please do the following:

 go to FILE / Export/Backup menu (from Dashboard or Portfolio)
Select “To share with others” export file protection option on top right
Leave the checkboxes already checked, and also check LOG FILE and QUOTE DATA boxes.
Export (note the location of the file – it is shown in center of the right side of that screen)

The file will be too large to email. Please send it to us using https://wetransfer.com/

Or one of the other cloud storage sites (dropbox, google drive, etc)

 

[Doug Hayman]

OK Jerry, I wetransfer'd it over .   About 600 MB MTExport file.

Thanks for looking into this.

[Jerry Medved]

you are using Renko charts with settings that result in huge # of bars when used with some symbols. If you specify the Box Size in % instead of absolute values, it may work better with various symbols. 

If you change the two Renko charts to 0.25% or 0.5% box size, you will see that everything switches better and probably uses less memory. Can then adjust from there.

[Doug Hayman]

The only problem with using % is that I cannot easily anticipate the addition of the next bar (and hence, the next trade).  Also, using % can/does reduce the # of bars, but also reduces the # of potential trading opportunities for me as well.

 

[Doug Hayman]

Further to last post, I'm looking to make this more efficient, without losing functionality.  Is there a way for me to continue to collect 120 days of data as I'm currently doing, but to force charts to only display last day or 2 only for example, so that MT doesn't have to buffer data for the full 120 days?   I guess I'm asking for a smart display filter where MT wouldn't have have to buffer all this data, but all the data is still there.

[Jerry Medved]

it is the combination of the 120 days and the low box size that is causing the slower switching of the charts. for Renko, MT has to compute all of it.

[Doug Hayman]

Yeah, I get it.   Maybe I'll just save SDATA.BIN to SDATA.BIN_BACKTEST once I hit close to 120 days of saved futures data, then subsequently reduce the # of saved days in Chart Config, save that current state of SDATA.BIN_CURRENT, and restore old SDATA.BIN_BACKTEST only when I do backtesting.   I could then restore current SDATA.BIN_CURRENT to SDATA.BIN when I go back to normal trading, and backfill to bring it up to date for # of days specified in Config.  I realize that I'll need to shut MT down, when doing these SDATA.BIN copying.

See any snags with that, other than mucking with SDATA.BIN?

Alternatively, would it be difficult or feasible to introduce a Settings switch that says to only buffer current day data for charts, even if there is more data for a ticker?

Thanks again.

[Jerry Medved]

a bit of a PIA but yes, would work.

As for separate settings, really don't want to separate the two