Sunday, November 22, 2015

Data Structures

Date Structures:

Data Structure in general means ‘Structure of Different Data Types’. The ILE RPG compiler allows you to define an area in storage and the layout of the fields, called subfields, within the area.

Data Structure can be defined by specifying DS in positions 24 through 25 on a definition specification.

Uses of Data Structure:

  •     Define the same internal area multiple times using different data formats
  •     Define a data structure and its sub fields in the same way a record is defined.
  •     Define multiple occurrences of a set of data.
  •     Group non-contiguous data into contiguous internal storage locations.
  •     Operate on all the sub fields as a group using the name of the data structure.
  •     Operate on an individual sub field using its name.

Special Data Structures:

  • A data area data structure (identified by a U in position 23 of the definition specification)
  • A file information data structure (identified by the keyword INFDS on a file description specification)
  • A program-status data structure (identified by an S in position 23 of the definition specification)
  • An indicator data structure (identified by the keyword INDDS on a file description specification).

Few Important Points about Data Structure:

    Data structures can be either program-described or externally described, except for indicator data structures, which are program-described only. One data structure can be defined like another using the LIKEDS keyword.

    A program-described data structure is identified by a blank in position 22 of the definition specification. The subfield definitions for a program-described data structure must immediately follow the data structure definition.

    An externally described data structure, identified by an E in position 22 of the definition specification, has subfield descriptions contained in an externally described file. At compile time, the ILE RPG compiler uses the external name to locate and extract the external description of the data structure subfields. You specify the name of the external file either in positions 7 through 21, or as a parameter for the keyword EXTNAME.

Date Formats in IBM i

Format name
Date-format parameter
Date format and separator
Field length
Job default

International Standards Organization
IBM® USA Standard
IBM European Standard
Japanese Industrial Standard Christian Era

What is Subsystem?

What is Subsystem?

Subsystem is an operating environment, defined by subsystem description, where the system coordinates processing and resources.

These can be used to control how different jobs run on your system and how much resources are allocated to different jobs.

Here are simple examples on how subsystem can be used.


For instance – in a mostly interactive environment you may want all your batch jobs to run in a batch subsystem which has 25% of your systems memory allocated to it, while your interactive jobs run in a subsystem which has 75% of your systems memory allocated to it. In a mostly batch processing environment you may want more memory allocated to your batch jobs and less for your interactive.


If you have several customers that use your AS/400 with different agreements on how many users can access the system – you can allocate subsystems based on device IDs and you can use different subsystems to limit the number of active jobs per subsystem. So if all customer A’s workstation devices start with an A, all their interactive jobs would go to subsystem A, etc.

Each Subsystem has its own run time attribute that is used by the job to have it executed under that environment.

Subsystems would generally classify based on the Type of Jobs that are being run. Subsystem can also handle different types of Jobs.

Below are the few simple IBM Supplied Subsystems.

1. QBASE - Supports Interactive, Batch and Communication Jobs.

2. QSPL - Spool Subsystem that supports reader/writer Jobs.

3. QCTL - Controlling Subsystem that starts up the system console.

4. QINTER - Supports Interactive Jobs.

5. QBATCH - Supports Batch Jobs.

6. QCMN - Supports all communication Jobs.

How to Create Subsystem?

We can create our own subsystems to run interactive/batch jobs.

We can start or end subsystems whenever we need, without disturbing our other system jobs (Jobs running under QBATCH/QINTER/Our own Subsystem).

Basic elements needed to create subsystem.

a. Subsystem Description

b. Job Queue

c. Class Object

d. Routing Entries

Steps to Create Subsystem:

Create Subsystem Description (CRTSBSD):

        POOLS((1 *BASE))                
        TEXT('My subsystem description')

Subsystem ‘TESTSBSD’ will run in Pool ‘1’ and will use *BASE for main memory storage.

There can be maximum of 50 Jobs inside the subsystem at a time.

Create Job Queue (CRTJOBQ):


Add Job Queue Entry (ADDJOBQE):

Once the JOBQ is created, We need to attach the JOBQ to subsystem.


MAXACT denotes the number jobs that can be processed at a time under this JOBQ.

Create Class (CRTCLS):

Class defines the run time attributes like, Run priority, Time Slice, Default wait time, Maximum temporary storage etc.


Add Routing Entry (ADDRTGE):


Start Subsystem (STRSBS):


Once the Subsystem is started, it can be used to submit the jobs.

End Subsystem (ENDSBS):

Subsystem can be end by using ENDSBS command.


Debug Batch Job

When you are working with Production Support, You will get a scenario where you would need to debug a job for analyzing unforeseen errors.

Here are few simple steps for debugging a Batch Job.

1. Service the Specific Job by using STRSRVJOB Command.

STRSRVJOB JOB(<Job Number>/<Job User>/<Job Name>)

2. If the Batch Job is currently running, You can check the Call Stack to see what program is currently running.


Option – 11 to See Call Stack.

3. Start the Debug of the program that is currently running.


Job will change to debug mode and will show the source in debug whatever the step that is currently being executed.

If you are submitting a new job and would like to debug it. Below are the steps.

1. Submit the job with HOLD(*YES) parameter on SBMJOB Command. Or, you could consider

HLDJOBQ <JOBQNAME> and Submit the Job.

2. Retrieve the Job Details and Service the Specific Job by using STRSRVJOB Command.

STRSRVJOB JOB(<Job Number>/<Job User>/<Job Name>)

3. Start the Debug of the program that is currently running.


4. Release the Job. Press F10 to set break point by going through DSPMODSRC.

Logical File vs OPNQRYF

1. The main difference between Logical file and OPNQRYF is that, Logical file is a permanent object and OPNQRYF is not. Therefore a logical file almost always (depending on your AS400 setup) will have usable access path to sort the records. i.e. access path will be permanent for a logical file.

2. An OPNQRYF will always need to determine what access path to use based on the sort order and filtering set in the command call. If an access plan exists that it can use it will, otherwise it will copy the data and do what is called a table scan to find the records.

3. QRYSLY keyword can be used on OPNQRYF command to setup Record Selection process. Key fields can be mentioned on a File using KEYFLD keyword.

4. Since a LF is a permanent object it will incur permanent storage needs which will also affect overall system performance.

5. If you are always going the same sort order and sequence again and again, then a LF is the best way to go, because it will be almost a readily available access path. It will save the processing time.

6. Now if you need a multiple sort order and random filtering, then maybe you should use OPNQRYF, but performance will be an issue.

7. Another aspect to consider is interactive updating. Persistent Access paths need to frequently update to be useful. Typically the best time for this to happen is when a record gets added or updated.

8. This of course affects performance, especially with large record sets that need to be reordered/ recalculated for each update. Especially when specific logical file doesn’t need to be updated for the process.

9. DYNSLT keyword on Logical File provides an advantage of performing a Selection Criteria during initialization time of program. This method is preferred when there is limited use of specific Logical File.

10. OPNQRYF would be the best option when you have database files with smaller amount of data and you have multiple selection criteria to be applied or criteria can be changed by job.

Service Program, Binder Source & Binding Directory

Service Program:

A service program (object type *SRVPGM) is a set of procedures that can be bound into ILE RPG programs. Basically when the service program is referenced any of its procedures might then be called. These callable procedures that are defined to the service program are referred to as exports which can be any ILE module, program, or even other service programs.

Service programs do not have a program entry procedure and therefore are non-callable. When bound the service program code is only referenced by linkages. That is, unlike other bound programs, the service program exports are not directly copied into the compiled program. The benefit of using a service program is easier maintenance, consistent performance, and more efficient processing.

As you know many software packages will have some global APIs. A date manipulation/validation API is a prime example of when a service program, full of date validation and conversion sub procedures, could be used. Other common examples include string manipulation procedures, combining procedures that retrieve or manipulate the run-time environment (adding to the library list for example), message handling, authority checking, XML, data encryption, IFS interface, email, etc.(something like a Java class or windows DLL).

RPG Modules can be created using CRTRPGMOD. A Module can have much number of procedures. These procedures are as similar to reusable subroutines, only more flexible in view of passing parameters, returning result variable. Starting and ending lines of the procedure can be identified with Begin & End of Procedure Statements on P - Spec.


You create a service program, and bind modules to it, with CRTSRVPGM which does not require a source member.


Binder Source:

By using EXPORT(*ALL) on CRTSRVPGM Command, Service program will be created with all procedures available in attached Modules.

You will find the importance binder source when there are many procedures in modules being attached and you would like only few modules to be exported through service program, You can achieve that by using binder source.

Example source for Binder Source.

             STRPGMEXP  PGMLVL(*CURRENT)                            
             EXPORT     SYMBOL(PROCEDURE1)                          
             EXPORT     SYMBOL(PROCEDURE2)                          


Binding Directory:

Exporting procedures from service program is powerful tool in developer tool kit, yet you can gain more advantages of it by using Binding Directory.

E.g. 1:

You are writing a service program to export modules from 10 different Modules. And, few of these modules are using procedures that are being exported by different service program.

In this case, You can mention the Modules & Service programs in Binding Directory and use CRTSRVPGM with parameter BNDDIR(BIND1).


E.g. 2:

You are writing an RPGLE program which would take advantage of procedures that have been created in multiple service programs.

You can take advantage of Binding directory by attaching those service programs to binding directory and attach it to your program.

Referring to the service program can be done one program at a time or, better yet, by using a binding directory. With the binding directory specified in the control specifications any unresolved procedure, sub procedure, program, or service program in that directory can be accessed when compiling and the CRTBNDRPG, CRTPGM, and CRTSRVPGM will not need individual module names specified. To see an IBM binding directory use the following command;


After the binding directory has been created and procedures have been added the calling program can specify the binding directories in the H specs ;

0002.00 H BNDDIR('BIND1':'BIND2')

Such a program must be created with the CRTBNDRPG command.

Note that the DFTACTGRP is set to *No to take advantage of ILE binding. Think of DFTACTGRP(*YES) as more related to OPM programming and DFTACTGRP more appropriate for ILE.

There are a few issues to consider when using service programs. When the caller references the service program some overhead is incurred. Normally this is offset by performance improvements. It is only an issue if the service program is called from the same program repeatedly. Also, certain changes to service programs can alter the "signature" of the service program which might require rebinding or recompiling some programs. This situation can be avoided by using binding source. 

Journals & Retrieve Data from Journals

Journaling on IBM i:

On IBM i, Journaling means process of recording an object's activity.

When it comes to journaling of Physical file, It records Open/Close/Write/Update/Delete Operations on Physical file.

How to Start Journaling:

1.      Use the Create Journal Receiver (CRTJRNRCV) command to create a journal receiver.

The journal receiver is where the journal entries are actually recorded. The journal "connects" the receiver to the file.

Also, you should put journal receivers in the same library as the file.

2.      Create a journal. Use the Create Journal (CRTJRN) command to create a journal and specify the receiver created in step 1.

Although you can journal multiple files to the same journal, you will generally want to have a journal "serving" a single file.

3.      Start journaling the file. This is done by using the Start Journal Physical File (STRJRNPF) command.

This is how you associate a file to a journal. Once the association is made, the system will record in the journal receiver a copy of any record added, updated or deleted from the file.

Other activity, such as when the file is opened and closed, can also be recorded in the journal receiver if you choose by selecting the appropriate options on the STRJRNPF command.

Retrieve Data from Journal:

You would get a scenario where records were deleted wrongly or update happened wrongly, You would need to retrieve the data back.

Below are the steps to be followed in order to retrieve data from Journal.

1.     Issue DSPJRN Command by specifying Journal Name & Journalled Physical File Name.

You might want to consider giving specific Journal Receiver Name. By default it would be current Receiver.

2.    Create a Copy of Physical File to include Journal Fields along with Original Physical File Fields.

3.    Use CPYF Command with FMTOPT(*NOCHK) from DSPJRN Outfile to JR file with Journal fields.


DDS of Physical File (TESTFILE):

     A          R TESTFR                                                        
     A            TSFLD1        10A                                             
     A            TSFLD2        10A                                             
     A            TSFLD3        10A                                             
     A            TSFLD4        10A                                             

DDS of JR File with Journal Fields (JRTESTFILE):

     A          R JRTESTFR                                                      
     A            JOENTL         5S 0                                           
     A            JOSEQN        10S 0                                           
     A            JOCODE         1A                                             
     A            JOENTT         2A                                             
     A            JODATE         6A                                             
     A            JOTIME         6S 0                                           
     A            JOJOB         10A                                             
     A            JOUSER        10A                                             
     A            JONBR          6S 0                                           
     A            JOPGM         10A                                             
     A            JOOBJ         10A                                             
     A            JOLIB         10A                                             
     A            JOMBR         10A                                             
     A            JOCTRR        10S 0                                           
     A            JOFLAG         1A                                             
     A            JOCCID        10S 0                                           
     A            JOINCDAT       1A                                             
     A            JOMINESD       1A                                             
     A            JORES          6A                                             
     A            TSFLD1        10A                                             
     A            TSFLD2        10A                                             
     A            TSFLD3        10A                                             
     A            TSFLD4        10A                                             

Issue DSPJRN Command on TESTFILE:


Use CPYF Command to get formatted Output of file data.


You would need to check the scenario where you need to recover the data and can query based on requirement.

Advantage of Using Sub-Procedures over Sub-routines in RPGLE

Need for Sub-Procedures or Sub-Routines:

The main advantage of using subroutines is intellectual manageability. That is, I can think of a program as a group of related small tasks, all of which are simple enough for me to understand, rather than as one gargantuan task that is more than I can handle. This gives ease in modifying/bug fixing of a program.

Debugging seems to be easier because I can often determine which subroutine most likely contains an error. Finding a logic error in a program of sub-procedures/sub-routines is easier when compared against the program which has complete logic in single routine.

Sub-Procedures/Sub-Routines also promotes the re-usability of code.

Limitations of Sub-Routines:

1.     Sub-Routines use global variables only.

That is, any variable or constant or indicator that I use in a subroutine may be used anywhere else in the program--in the main calculations, in any subroutine, in output specs. This can lead to undesirable side effects. Changing the value of a variable in one part of the program causes something to go wrong in another part of a program.

So, copying the code of sub-routine to different program would need changes to be done in view of variables.

2.     Inability to use Parameters.

If we have a code that performs specific calculation based on input parameters, we would need to initialize the variables before calling the program.

Advantages of Sub-Procedures:

1.     Sub-Procedures allow you to define local variables.

Local variables would be allowed only inside sub-procedure. So, there is no risk by using the variables with same name in other sub-procedures or main line. These variables cannot be referenced outside of the program.

2.     Ability to use parameters.

Parameters make sub-procedures more portable by providing internal names for required data. It makes program maintenance easy when the procedure is being called from multiple routines.

3.     Returning a Value

Sub-procedure has the ability to return a value. It can be used as how BIF are being used on operations like EVAL, IF etc.

4.     Ability to use Sub-Procedure across application.

If we have Sub-Procedure that will be useful across application, It can be attached to Service Program and made available to access across application.

Track the Submitted Job Details

If you are using CL Program to Submit the Jobs, You can be able to Track the Details by using RCVMSG Command.


The Receive Message (RCVMSG) command is used by a program to receive a message previously sent to a message queue.

The RCVMSG command receives messages from a job message queue (a message queue associated with a call stack entry or the external message queue (*EXT)), or from a named message queue. The program can receive a message from a message queue associated with its own call stack entry or from a message queue associated with another call stack entry.

This command copies a message received in the specified message queue into control language (CL) variables within the program.

You can specify the message being received by indicating the message type, the reference key of the message, or both. The program receiving the message can also specify, on the RCVMSG command, whether a message is removed from the message queue or left there as an old message. If the specified message queue is not allocated to the job in which this command is entered, or to any other job, the message queue is implicitly allocated by this command for the duration of the command's processing.

Example Program to Retrieve Job Details:

Below CL Program will show on how to retrieve Job Details (Job Name/Job User/Job Number)

             DCL        VAR(&MSGDTA) TYPE(*CHAR) LEN(80)                        
             DCL        VAR(&JOBNAM) TYPE(*CHAR) LEN(10)                        
             DCL        VAR(&JOBUSR) TYPE(*CHAR) LEN(10)                        
             DCL        VAR(&JOBNUM) TYPE(*CHAR) LEN(6)                         
             DCL        VAR(&JOBSTS) TYPE(*CHAR) LEN(10)                        
             SBMJOB     CMD(DSPSYSSTS) JOB(DSPSYSSTS)                           
             RCVMSG     RMV(*YES) MSGDTA(&MSGDTA)                               
             CHGVAR     VAR(&JOBNAM) VALUE(%SST(&MSGDTA 1 10))                 
             CHGVAR     VAR(&JOBUSR) VALUE(%SST(&MSGDTA 11 10))                 
             CHGVAR     VAR(&JOBNUM) VALUE(%SST(&MSGDTA 21 6))                  

Load All Subfile


Subfile is one of the most used concept in IBM i.

There are three different types of Subfiles.
  • Load All Sub File
  • Expandable Sub File (Click Here to see more about Expandable Subfile with an Example)
  • Single Page Subfile
Message Subfiles are used to display the messages on the screen. Click Here to see more about Message Subfiles.

We will see about Load All Subfile in this post. Click Here for Full Free Format RPGLE code for this example. 

Load All Subfile:

In Load-All subfile all the records are written to the subfile buffer at once and then the data in the subfile loaded from the buffer.
In this case SFLSIZ should be at-least 1 greater than the SFLPAG.
If we are writing more records in the subfile buffer than the SFLSIZ declared and SFLSIZ<9999, then the SFLSIZ is extended to accommodate all records till the size of 9999. 9999 is the buffer limit.
In this case PAGEUP AND PAGEDOWN is taken care by system. SFLPAG should be less than SFLSIZ.
In load-All subfile if we do PAGEDOWN and then press ENTER on the page, by-default ENTER bring the display screen to the very first page irrespective of the current page number. To avoid this situation, we use file information data structure to get the current page RRN number and pass it to the SFLRCDNBR hidden field defined in the display file DDS.


Sample program below will load records from Physical file to Load all subfile. Output will be shown as below.


 TESTLODALL               Test Load All Subfile                         10/09/14
   Display Records for Load All Subfile                                         
  Test Field1       Test Field2      Test Field3        Test Field4             
  -----------       -----------      -----------        -----------             
  TESTREC17         TESTREC17        TESTREC17          TESTREC17               
  TESTREC18         TESTREC18        TESTREC18          TESTREC18               
  TESTREC19         TESTREC19        TESTREC19          TESTREC19               
  TESTREC20         TESTREC20        TESTREC20          TESTREC20               
  TESTREC21         TESTREC21        TESTREC21          TESTREC21               
  TESTREC22         TESTREC22        TESTREC22          TESTREC22               
  TESTREC23         TESTREC23        TESTREC23          TESTREC23               
  TESTREC24         TESTREC24        TESTREC24          TESTREC24               
  F3=Exit  F12=Cancel                                                           

Sample Code for Physical File (TESTFILE)

     A          R TESTFR                                                        
     A            TSFLD1        10A                                             
     A            TSFLD2        10A                                             
     A            TSFLD3        10A                                             
     A            TSFLD4        10A                                             

Sample Code for Display File (TESTSUBFIL)

     A*%%TS  SD  20141010  053209  USERNAME    REL-V6R1M0  5761-WDS             
     A                                      DSPSIZ(24 80 *DS3)                  
     A          R SFDTAR                    SFL                                 
     A*%%TS  SD  20141010  053209  USERNAME    REL-V6R1M0  5761-WDS             
     A  70                                  SFLNXTCHG                           
     A            SFTSFLD1      10A  O 10  3                                    
     A            SFTSFLD2      10A  O 10 21                                    
     A            SFTSFLD3      10A  O 10 38                                    
     A            SFTSFLD4      10A  O 10 57                                    
     A          R SFCTLR                    SFLCTL(SFDTAR)                      
     A*%%TS  SD  20141010  053209  USERNAME    REL-V6R1M0  5761-WDS             
     A                                      CA03(03)                            
     A                                      CF12(12)                            
     A                                      PAGEDOWN(50)                        
     A                                      PAGEUP(51)                          
     A                                      OVERLAY                             
     A  25                                  SFLDSP                              
     A  26                                  SFLDSPCTL                           
     A  27                                  SFLINZ                              
     A  28                                  SFLCLR                              
     A  40                                  SFLEND(*MORE)                       
     A                                      SFLSIZ(9999)                        
     A                                      SFLPAG(0008)                        
     A                                  1 27'Test Load All Subfile'             
     A                                      COLOR(WHT)                          
     A                                  1 73DATE                                
     A                                      EDTCDE(Y)                           
     A                                      COLOR(BLU)                          
     A                                  2 73TIME                                
     A                                      COLOR(BLU)                          
     A            SFPGMNAM      10A  O  1  2                                    
     A                                  7  3'Test Field1'                       
     A                                  7 21'Test Field2'                       
     A                                  7 38'Test Field3'                       
     A                                  7 57'Test Field4'                       
     A                                  8  3'-----------'                       
     A                                  8 21'-----------'                       
     A                                  8 38'-----------'                       
     A                                  8 57'-----------'                       
     A                                  5  4'Display Records for Load All Subfi-
     A                                      le'                                 
     A                                      COLOR(BLU)                          
     A            CSRRRN         4S 0H      SFLRCDNBR(CURSOR)                   
     A            SFRRN          4S 0H                                          
     A          R SFFTRR                                                        
     A*%%TS  SD  20141010  024314  USERNAME    REL-V6R1M0  5761-WDS             
     A                                 23  3'F3=Exit  F12=Cancel'               
     A                                      COLOR(BLU)                          

Sample RPGLE Program for Loadall Subfile (TESTLODALL): Click Here for Full Free Format RPGLE code for this example.

       // Define Physical File                                                                     
     FTESTFILE  IF   E             DISK                                                            
       // Define Subfile                                                                            
     FTESTSUBFILCF   E             WORKSTN SFILE(SFDTAR:SFRRN)                

       // Define Required Variables

     C                   ExSr      Sub_Init                                   

     C                   ExSr      Sub_Main                                   

     C                   ExSr      Sub_Exit                                                         

       // Sub_Init - First time initialization routine                        

     C     Sub_Init      BegSr                                                

     C                   Eval      SFPGMNAM = 'TESTLODALL'                    

       // Clear Subfile                                                       

     C                   Eval      *In25 = *Off                               
     C                   Eval      *In26 = *Off                               
     C                   Eval      *In28 = *On                                
     C                   Write     SFCTLR                                     
     C                   Eval      *In28 = *Off                               
     C                   Eval      *In25 = *On                                
     C                   Eval      *In26 = *On                                

     C                   EndSr                                                
       // Sub_Main - Main Processing Routine                                  
     C     Sub_Main      BegSr                                                

     C                   ExSr      Sub_Load                                   
     C                   Dow       *In03 = *Off                               
     C                             And *In03 = *Off                           

       // If You Need Cursor to be Pointed on Specific Record,                    
       // You can move corresponding RRN to CSRRR field. In this                  
       // example I am placing Cursor on 3rd Page.                                
     C                   Eval      CSRRRN = 17                                    
     C                   If        SFRRN = 0                                      
     C                   Eval      *In25 = *Off                                   
     C                   EndIf                                                    
     C                   Write     SFFTRR                                         
     C                   Exfmt     SFCTLR                                         

     C                   EndDo                                                    

     C                   EndSr                                                    
       // Sub_Load - Load Subfile                                                 
     C     Sub_Load      BegSr                                                    

     C                   Eval      SFRRN  = 0                                     
     C                   Eval      CSRRRN = 1                                     
     C                   Read      TESTFILE                                       
     C                   Dow       Not %Eof(TESTFILE)                             
     C                             And SFRRN < 9999                               
     C                   Eval      SFRRN = SFRRN + 1                              
     C                   Eval      SFTSFLD1 = TSFLD1                              
     C                   Eval      SFTSFLD2 = TSFLD2                              
     C                   Eval      SFTSFLD3 = TSFLD3                              
     C                   Eval      SFTSFLD4 = TSFLD4                              
     C                   Write     SFDTAR                                         
     C                   Read      TESTFILE                                       
     C                   EndDo                                                    
     C                   Eval      *In40 = *On                                    
     C                   EndSr                                                    
       // Sub_Exit - Finalization Routine                                         
     C     Sub_Exit      BegSr                                                    
     C                   Eval      *InLr = *On                                    
     C                   EndSr                                                                      

Different Ways of Sorting Data in a List - Python

Sorting Data in a List List is a collection of data (of different data types), much like an array. Like any data structure or data set, dat...