Improving MassTransit 4.2 log and file performance

1 Star2 Stars3 Stars4 Stars5 Stars
Loading...
  • Product:
    MassTransit
  • Version:
    4.2-Current
  • Document Type:
    Info
  • Revised:
    12/2/2003
  • Reviewed:
    12/2/2003

Summary:

MassTransit 4.2 includes changes that can improve performance when working with large sets of files and log entries. This change involves a tradeoff between performance and increased potential for database integrity issues.

Two of the changes are permanently set, but there is a configuration option that can be changed for the third one if needed.

Note: users who are upgrading from a previous version and maintaining their MassTransit databases will need to follow the steps below to take advantage of the speed improvements.

Description:

BACKGROUND:
MassTransit maintains information such as the log and list of contacts in five database files:
dCLIENT.dat
dFILES.dat
dJOB.dat
dLOG.dat
dVARDATA.dat

Each database file has a corresponding index file that ends in .idx instead of .dat, such as dLOG.idx.

Starting in version 4.1, a change was made to improve database integrity for all of the databases (technically we refer to the change as “WRITETHRU” being enabled).

DATABASE CHANGES AND OPTIONS IN MASSTRANSIT 4.2
A. Log and File Indexes: MassTransit 4.2 is now set to favor performance (WRITETHRU disabled) for the Files and Log databases indexes (dFILES.idx and dLOG.idx). This results in a significant performance gain. For example, clearing a log of 32000 entries could take over 30 minutes with the old approach and less than 30 seconds with the new approach. There is a slightly increased risk of corruption problems with the new approach, but if a problem occurs, simply delete dFILES.idx and/or dLOG.idx file. MassTransit will then re-create the index with no loss of data.

If you are upgrading from MassTransit 4.1.x to 4.2, you will need to perform the following steps to benefit from the performance gains:
1. Quit MassTransit
2. Delete the files dFILES.idx and dLOG.idx from the MassTransit Databases folder. These indexes will be re-created and no loss of data will occur.
3. If you would like to further increase log performance, you can delete the dLOG.dat file as well. Note that if you delete this file you will lose all current log entries.
4. Re-launch MassTransit.

B. Log Database: By default, MassTransit now favors performance for the log database itself, dLOG.dat. Since this affects the log database itself rather than the index as described above, you may choose to change the default to minimize the potential for database integrity problems with the log, at the expense of the additional speed option. To do so follow the steps below:

1. Find the file MassTransit.cfg in the Extras folder within the MassTransit installation folder.
2. Double-click the file to open it in TextEdit or another text editor.
3. Find the line LOG_WRITETHRU = FALSE and change it to LOG_WRITETHRU = TRUE
4. Save the file.
5. Move the file from the Extras folder to the main MassTransit folder.
6. Launch MassTransit (if it’s already running, quit and re-launch for the changes to take effect).

Tags: