Compress tick data on the fly or just drop the tick data every minute

[ahujat]

I guess the crypto, being tradeable 24/7, produces 5 times as much tick data as a stock(RTH). Can the tick data be dropped, every minute and then just have 1 minute OHLC? I don't think a lot of people care about tick data.

[Jerry Medved]

cannot do that for performance reasons

[ahujat]

How about doing that at night (user specified time), when the user is inactive?

Or like TWS, auto restart the app (again at user specified time) with data compressed as part of the startup.

[Jerry Medved]

that would not get rid of the requirement of changing all the data locking mechanisms and really REALLY don't want to copy TWS restart stuff. Hate it with a passion 

[ahujat]

But there has to be some better solution. Every couple of days, I have to open the dashboard, click on settings, click on charts->general. Enable 'consolidate data on next startup'. Then exit the application and then restart it. I was doing it once a week or in 10 days but with crypto, I have to do it every 2-3 days. On the surafce, it doesn't look like a big deal but it's just irritating when you have to do it frequently.

[Jerry Medved]

that's entirely different functionality and you do not need to do that. Just restart MT.

[ahujat]

Hmm. What does 'consolidate date on next startup' do then?

[Mike Medved]

It consolidates the data on next MT startup. Takes a little while to do so sometimes (could be a few minutes even).

[Jerry Medved]

that "consolidate" is basically if your data file grew a lot and now has a lot of empty space (like if you cleared data, or large amount of data expired), it creates a new data file and only copies active data to it, thus shrinking the file. Other than this, the data file never shrinks.

[stock777]

What I've been doing is doing a manual clear of crypto symbols, you can do the whole port in one shot

I assume that gets rid of all the tick data, if not from the database then at least from the data MT has to slog through

 

Edited November 22, 2021 by stock777

[Jerry Medved]

actually, I updated the OHLC consolidation - before it was only on startup and only if not done in a while. Changed it to be every 4 hours, but only if idle - idle being no action with menus or MT windows like clicking on them. will be in next beta.

[ahujat]

44 minutes ago, Jerry Medved said:

actually, I updated the OHLC consolidation - before it was only on startup and only if not done in a while. Changed it to be every 4 hours, but only if idle - idle being no action with menus or MT windows like clicking on them. will be in next beta.

Perfect.

No action for how long to be 'idle'? 

[Jerry Medved]

15 min