* This patch is replacing patch 763, which had serious performance
problems.

* It prevents the same issue, by preventing duplication of 'add' and 'check_and_split_data'
processes. This will avoid corruptions of the
database in some cases.

* Cleanup of database after crash or a similar situation.

The cleanup script is found under /opt/openit/contrib/bin/db_clean_structure.pl.

It can be activated to run regularly by making a file:

/var/opt/openit/etc/database_test_active

It will then run whenever the database is being updated by aggregation.

It can be run manually, and then it has to run as the "openit" user.

Revision 2 is for fixing a dependency problem with patch 782.

Supports removal of massive lists of classification values
in the WebGUI, to speed up processing and to make the
list easier to handle.

To activate it, set, in /etc/opt/openit/openit.cfg:

CLASSVAR_REGEXP_FILTER   $CONFIG/classvar_regexp_filter

Make the file, and enter into it:

<data type name>:<classification name>:<regexp>

for each data type / classification wanted.

When the regexp matches, the entry will be removed from
the classification list in the WebGUI.

Rev 2: Added requirement for patch 723.

check_and_split_data used to abort processing when encountering an olap file with illegal time
stamp. This has been fixed so that it will rename file to *.inconsistent and proceed with next file.

Revision 3: Added Perl modules for compatibility with older versions of Open iT.

Revision 4: Supersedes 561.

* Processing in chunks, avoiding multiple check_and_split_data processes.
* Handling of start/end time of days with DST changes.