Realtime Imports

Overview

Quickly identifying when changes occur in another database can be challenging.  Astra Schedule handles this by employing a realtime “differencer”.  The differencer runs as a separate service that constantly watches the SIS database for changes to certain fields.  This process compares field values in changed records to identify key field changes and then notifies the system that an update is required. A realtime import job may be configured and scheduled to run as often as desired to find changes in your SIS and update the corresponding records in Astra Schedule.

 

The realtime import process will insert new records when found, but typically updates fewer fields on existing records than batch imports (as defined by the import configuration files). This enables the process to work extremely quickly, keeping critical elements current during scheduling processes.  Depending on configuration, a batch import may still be required periodically to update certain fields.

 

Additionally, the realtime import process will not delete records from Astra Schedule that are deleted from the SIS.  The realtime differencer requires that records exist for comparison and action.  A batch import is required to delete records from Astra Schedule that no longer exist in the SIS.

NOTE: It is recommended that a batch import be performed for the term in question before activating a realtime import job schedule. The realtime import process assumes that records are synchronized at the point the job schedule becomes active, so will not detect differences until the records are again edited.

Realtime XML File

When setting up the realtime import process, a realtime SIS job is created.  During the configuration of the job, an xml configuration file is selected that specifies which subset of the data will be checked for changes.  By only comparing a few critical fields for changes, the performance of the realtime process can be extremely fast.

 

The realtime-specific xml import configuration file supplies the system with the information it needs to perform the update, including the SQL query used to get section information from the SIS and the specific fields being compared to identify changes.  The SQL query takes into account the interface job parameters for term and campus.  If applicable in the SIS data, the query can also take into account an activity date, against which it can compare changes in Astra Schedule.  If an activity date is available, this can further increase the speed of realtime updates.

 

Once the data is returned by the query, the fields specified by the configuration file for the difference check are compared to Astra Schedule.  The default fields are as follows:

 

 

If a difference is detected in one of these fields, the realtime process informs the updater process that certain section ID's require updating.  The realtime service's only job is to find the change and wake up the update process as needed.  The regular update process - the same one used to perform On-Demand updating - is utilized to perform the updated record import.  The fields included in the difference check may be edited to specify either more, or fewer, fields to be checked.  When the update is performed, the fields being updated are dictated by the configuration file being used by the on-demand update process.

Realtime Scheduling

The frequency at which the realtime import process runs is dictated by the job's schedule.  Job schedule settings can be configured to run as often as a few seconds if desired.  Multiple schedules can be configured per job to achieve realtime processing at different intervals at different times during the week.  For example, realtime could be scheduled to run every 15 seconds between 6 AM and 6 PM, Monday through Friday, but then only only once per hour during the evenings after 6.  Note that creating multiple schedules within a job that happen to overlap will not create a problem because each schedule runs independently on its own communication thread.  Realtime import is most commonly used to update with regular frequency during business hours during weekdays.  Realtime import job schedules can be enabled and disabled by changing the "is active" status flag.  To stop a job schedule from running, remove the flag to inactivate the schedule.  Add the flag to start the schedule again.  If desired, multiple realtime jobs can be configured and scheduled with varying term and campus parameter combinations.

 

Before enabling a realtime job schedule, it is necessary to run a batch job for the same term and campus to make sure the existing data is current.  The realtime process will begin checking for changes from the moment it is enabled forward, but will not catch differences that existed when it became live.  Additionally, when a realtime schedule stops and then restarts it doesn't know about changes that may have occurred in between.  Therefore, batch imports may still be required to update the system periodically with any changes that occur while realtime job schedule is dormant.  Best practice may be to schedule a batch import to occur at a down time, just before the realtime schedule begins.  For example, a scheduled batch import could run at 6 AM every weekday, and then realtime could begin at 6:30 AM and run until 6 PM every weekday.

 

The realtime process logs its activity in the Differencer folder in the installation directory.  This log file contains information regarding any errors encountered when attempting to connect to the SIS database as well as any other failures during the difference checking process.  During normal operation the system creates rows in the log file for each record queried and any changes encountered.

Setting up a Realtime Job

To configure the realtime import process, perform the following:

 

1.Click the Import/Export Management option on the Setup tab. The Job list page will be displayed.

 

2.Click the Add Job button.

 

3.Select the Real Time option from the drop-down list and click Add Job.

 

4.Enter a job name. This should be a name that will serve to remind you of the reason for the job and to help distinguish it from others on the list.

 

5.Enter a job description if desired.

 

6.Choose the appropriate "Differencer" xml configuration file from the drop-down list.
 
The database adapter type and data type supported by the configuration file selection are displayed for reference.

 

7.Select one or more terms that should be included in this job and use the left arrow button to add them to the Selected Items panel.

 

8.Select one or more campuses that should be included in this job and use the left arrow to add them to the Selected Items panel.

 

NOTE: The list of terms and campuses corresponds to the terms and campuses created in Astra Schedule. The SIS names of these items must match in order for the import of data to be successful.

 

9.Click Add Schedule to create a schedule entry for the job.

 

10.On the Job Schedule window, verify the active status of the job.

 

11.Configure a time frequency for the schedule.  The "Every" option is pre-selected for realtime import jobs.

 

12.Select and configure a day pattern for the schedule.

 

13.Specify the date range for the schedule.

 

NOTE:  The schedule being created will not start until the next start time in the cycle.  In other words if the schedule has a start time that is before the current time, the job will not start until the next day triggers the service to wake up.

 

14.Click OK to add the schedule to the Job Schedule list on the job form.

 

15.Click Save.

Page url: ?sisinterface_realtimeimports.htm