Data Retention parameters in Alaris SMS Platform

If you have ever wondered why some of the traffic you used to see in Analytics is no longer there, or why you can't export some of the EDRs either via the EDR export tool or EDR-related reports, and you're sure that the traffic was certainly there and processed, this article on data retention will help you out.

Traffic retention: EDRs and analytical cubes

EDRs are the primary source of traffic data that the system works with, builds and displays all statistics, and provides access to traffic-related data in the system. EDRs are aggregated into more grouped versions of themselves called analytical cubes - there may be minute cubes, they originate hour cubes, day cubes are created from the hour cubes, the day cubes give rise to week and month cubes, and the year cubes come from month cubes.

Alaris SMS Platform has an EDR retention mechanism which operates according to the system settings (Start/Administration/System settings) that define the periods for which the system stores EDRs in active and archive database partitions:

• Active EDR day count
• Archive EDR day count

The active partition contains EDRs that are being used in the system and its REST API to display and access all kinds of traffic-related information, such as Analytics, Reports, EDR export tool, EDR rerating, Invoices. The archive partition stores EDRs that are no longer in active use by the system and can only be restored for viewing upon request to Alaris Labs support team. It's not recommended to change the values of any of these parameters, as it must be evaluated and calculated beforehand, as the change may require additional resources.

Analytical cubes, the EDRs that have been grouped and aggregated into larger chunks of data, are also retained. Each type of cube has their own system parameter which is responsible for its storage. They are:

• Minute cube partition count: number of minute cubes stored in the System; in other words, a period of time (in minutes) during which the System retains aggregated statistics arranged by minute cubes.
   
• Hour cube partition count: number of hourly cubes stored in the System; in other words, a period of time (in hours) during which the System retains aggregated statistics arranged by hourly cubes.
   
• Day cube partition count: number of daily cubes stored in the System; in other words, a period of time (in days) during which the System retains aggregated statistics arranged by daily cubes
   
• Week cube partition count: number of weekly cubes stored in the System; in other words, a period (in weeks) during which the System retains aggregated statistics arranged by weekly cubes.
   
• Month cube partition count: number of monthly cubes stored in the System; in other words, a period of time (in months) during which the System retains aggregated statistics arranged by monthly cubes.
   
• Year cube partition count: number of years for storage of year cubes.

GDPR: EDR and personal data contents retention

In some countries it is required to follow the EU GDPR policy. The good news is that the platform already supports some of the capabilities that enable its implementation, such as:

• not saving message text (so that it is not stored in the System): specify the list of client channel IDs in the System parameter Ignore message texts for customer channels (comma separated ID list). Applicable to both MO and MT traffic types. If a vendor channel ID is specified (as well as any dummy IDs or invalid values), it will be ignored by the SMS switch. Up to 2000 bytes can be specified. The allowed format is spaceseparated,
  comma-separated or space+comma-separated, for example: 23317 25370 23897 or 23317, 25370, 23897.
• masking destination addresses and texts: use SMS/EDR management/EDR masking interface.
• erasing message text after some time passed: configure the System parameter Wipe SMS contents after, days. The erasing job will be launched every day and texts of messages with timestamps older than the specified amount of days will be removed
• removing personal data off your inactive colleagues' users: refer to the System parameter Wipe personal data after, days of inactivity. Once the erasing job is launched, the following data will be replaced by the Info wiped by EU GDPR law record:
  • contact emails (Default invoice emails, Default rate change emails, Default technical emails, Account alert emails) from Carriers/Agreements for all agreements that were closed more than N days ago (End date field)
  • First name, Last name, Email fields from the Contacts section (except for Zip code) from Administration/Users in case the user’s carrier has no active agreements and all past agreements were closed more than N days ago. See also the Alaris YouTube video
• hiding message texts from specific users in the main (admin) web interface: contact Alaris Labs technical support team, share the code BZ50378 and provide the list of user names for which it must be enabled.

Other

• Log store period, days: the period to store the logs of different components (Carriers, Products, Routing rules, etc). The default value is 90.
  • Additionally, the parameter regulates how many days EDR autorerating tasks are stored. Note that only the last 10,000 records are stored even if the period has not been reached.
  • The setting also defines how many days other tasks (EDR export, Rate export) along with SMS channel status changes (DB table sms_channel_status_change) and EDR file statuses are stored in the System.
    
    The logs contain details of every change in System tables as well as the author of the changes. This allows investigating any incident (System breakdown, errors, malicious attacks, etc) However, logs take disk space and impact the DB size. Therefore, prior to modifying this parameter, consult Alaris Labs technical support team.

Rates

• Rate change log, days: defines storage period for rate change logs in the System. The default value is 90.
• Rate import files retention period, days: number of days during which rate import tasks (Rate import interface) and files of auto rate import (Administration/Email processing rules, Files tab) are stored to be shown in the System. Note that the storage period of files is also defined by the cron task located on the server (default: 30 days). To verify the internal value, contact Alaris Labs technical support team. The period to store files is defined by the minimum of the cron value and the System parameter’s value.
• Rate import analysis keep period, days: number of days during which the results of analysis (rate import tasks launched in Mode: Choice or Auto) are stored in the System (available tasks can be checked on the bottom grid of the Rate import interface by clicking on the view hyperlink). The default value is 7. Tasks created older than 7 days from the current date cannot be reviewed. If the parameter is set to 0, it will be possible to check only one last task (once a new one is created, all other tasks will become unavailable).

Reports

• Report task result storage period, seconds: the period to store report results to be shown in the web interface (available through the view hyperlink in Reports). The default value is 172800; it can be overridden on the report level.
• Report task storage period (days to store tasks): the number of days to store tasks created in Reports. The parameter functions along with the Report tasks count to store parameter. The default value is 30.
• Report tasks count to store: the number of report tasks created in Reports to be stored. The parameter functions along with the Report task storage period (days to store tasks) parameter. The default value is 10000.
• Report tasks count to store (per single report): the number of tasks within the same report to be stored in Reports. The default value is 100.

AKBSMS - Alaris Knowledge Base

Related Questions:
data retention
storage period
traffic storing
EDR storing
where is old data in analytics
report doesn't show several months old data

Link to this article: https://helpdesk.alarislabs.com/en/knowledge_base/article/315/category/144/