Monday, January 9, 2023

Removing elements from a list in Python with list method pop()

Removing elements from a list

It's often essential to remove the data from any data set (let's take 'list' in this case). 

The list method "pop()" comes handy in this case, this method allows removing the last element of a list and return the same. This is a very useful method for amending lists by removing it's elements and can be used in a variety of situations as required.

Using "pop()" to remove the last element from a list

By default "pop()" method removes the last element from a list and returns the same. 

One thing to note here is, it modifies the original list. 

1

2

3

4

5

numbers = [1, 2, 3, 4]

removed_element = numbers.pop()

print(removed_element) # 4

print(numbers) # [1, 2, 3]

 


In the above example, 
  • pop() method removes the last element (i.e., "4" in this case) from the list "numbers". 
  • Returns the removed element, in this case returned element is stored in "removed_element". 
  • Original list is modified to only contain the remaining elements. 
One thing to note here is, if we try to remove the elements from an empty list, it'll raise an IndexError.

1

2

3

4

5

numbers = []

removed_element = numbers.pop() # throws error

print(removed_element)

print(numbers)

 


When we run the above code to remove an element from empty list, an error would be throws (see below). 

Traceback (most recent call last):

  File "C:\Users\pradeep.panga\PycharmProjects\pythonProject\pop.py", line 2, in <module>

    removed_element = numbers.pop()

                      ^^^^^^^^^^^^^

IndexError: pop from empty list


It is important to handle such errors when working with "pop()" method to avoid program failure run time. 

This can be done in one of the two ways. 
  • By checking if the list is empty before calling the pop() method. 
  • By using the pop() method inside try-except block and monitor for the IndexError. 

Removing elements from specific positions in a list

When working with data, we do not always want to remove a last element from list. We often need to remove a specific element from the beginning or middle of a list. 

"pop()" can also be used to remove the element from any position. To achieve this, we would need to pass the index position of the element which needs to be removed.

1

2

3

4

5

numbers = [1, 2, 3, 4]

removed_element = numbers.pop(2)

print(removed_element) # 3

print(numbers) # [1, 2, 4]

 


In the above example, 
  • "pop(2)" removed the element "3" from index position "2". 
  • Original list has been amended to remove the corresponding element from the list. 

Using pop() in a loop to remove all elements from a list

In the previous examples, we have seen how to delete a single element from a list. 

We could use the "pop()" method in a while loop to remove all the elements from a list. 

E.g.: 
Removing all elements from a list by using pop() in a while loop. 

1

2

3

4

5

6

7

8

9

numbers = [1, 2, 3, 4]

removed_elements_list = []

while numbers:

   removed_element = numbers.pop()

   removed_elements_list.append(removed_element)

 

print(numbers) # []

print(removed_elements_list) # [4, 3, 2, 1]

 


In the above example, 
  • Line - 3: "while numbers:" - This loop is repeated until the list is empty. So, loop will not run on a empty list and IndexError will not occur. 
  • Lines - 4 & 5: Last element from the list is removed and added to the new list created (i.e., removed_elements_list).
  • Line - 7: Original list (i., numbers) is empty as all the elements were removed. 
  • Line - 8: New list (i.e., removed_elements_list) contain the elements in reverse order to the original list as the elements are removed from the last index and added. 
Using a while loop and pop() method to remove all elements from a list can be an effective approach, especially if the list is large. It avoids the need to create a new list, which can be slower and requires allocation of further system resources.

One thing to remember here is, This would only work if we do not require the original list as the pop() methods removed the element and by the end of the loop, original list would be empty. 

We could also keep a copy of the list if required, before starting to use the pop() method.

Conclusion

In this post, we explored the usage of a list method "pop()" and how to remove the element from the end of a list, any index position of a list and to remove all the elements by using pop() in a loop. 

Hope this has been a bit of help in understanding the use of list method pop() in Python.


If you have any Suggestions or Feedback, Please leave a comment below or use Contact Form.

Tuesday, January 3, 2023

Working with Multi-Member Physical Files - IBM i

Multi-member Physical Files

At times, Multi-member physical files are very helpful in IBM i.

A physical file is a file that stores data in a specific format. A physical file can have multiple members, each of which is act as a separate data set within the file. 

This allows us to store different types of data (format of the file remains same) or data for different purposes within the same file.

Creating a Multi-member Physical File

Create a physical file (PF) with single member (default) and add an addition member after PF is created.

A physical file is created by using CRTPF command.

CRTPF FILE(<Library>/<File>) MBR(<Member Name>)

By default, Member is created with the same name as the file name. If a specific name is required, member name can be added as well. 

CRTPF FILE(LIBRARY/FILE) RCDLEN(10) MBR(MEMBER1) TEXT('Temporary File')

For the simplicity, I have created the flat file by directly mentioning the record length (RCDLEN) as 10. We would usually be creating physical file based on the DDS source. 

Once the physical file is created, additional members can be added to the PF by using ADDPFM command. 

ADDPFM FILE(LIBRARY/FILE) MBR(MEMBER2) TEXT('Additional Member')

One thing to note here is that, by default PF is created with maximum members as '1'. 

We should either set this to the number of members required or set to *NOMAX to allow adding any number of members. 

This can be done either at the time of PF creation (i.e., using CRTPF) or after the PF creation (using CHGPF). 

CHGPF FILE(LIBRARY/FILE) MAXMBRS(2)

DSPFD (Display File Description) command can be used to view the maximum number of members allowed and the current members of the file. 

Accessing data from Multi-member Physical File

There are different ways of accessing the data from multi-member physical file depending upon from where we are accessing the data. 

From Command line/CL Program

Data in multi-member PF can be viewed by running the commands like DSPPFM (Display Physical File Member) and RUNQRY (Run Query). 
  • DSPPFM gives an option to enter the member name from which we need to view the data. 
    • By default data from the first member is displayed (*FIRST against MBR keyword). 
    • All records are displayed by default. We could specify the record number if we only need records from specific record number. 
  • On the other hand RUNQRY gives more flexibility for the record selection by the values of the fields with in the PF. 
    • By default data from the first member is displayed and can enter the member name as required. 
    • Field names would be prompted on selecting '*YES' against Record selection. 
Above two commands are mostly used to view the data from command line and not for processing the data programmatically. 

Other way to access the data in a multi-member PF is we will need to Override the file to a specific member and use the Open Query File (OPNQRYF) command or the Open Data Path (ODP) API. 

This would allow specifying the member of the physical file that we need to access and the conditions for selecting the data.

E.g.: 

We could use the following OVRDBF & OPNQRYF commands to override and select all records from the MEMBER1 member of the MYFILE physical file. 

OVRDBF FILE(MYFILE) TOFILE(QTEMP/MYFILE) MBR(MEMBER1)

There are few other important keywords to OVRDBF allowing us to specify the override scope, if the open data path to be shared and few other options. 

OPNQRYF FILE((MYFILE MEMBER1))

This would allow the user to access the data from the specific file member using the file name directly. 

There are few other important keywords to OPNQRYF allowing the user to query the data and if the file is opening for read only or allows update and delete. 

If we need to filter the records where the value of the MYFIELD field is greater than 10,

OPNQRYF FILE((MYFILE MEMBER1)) QRYSLT('MYFIELD > 10')

This file could then be used in the further CL and/or RPGLE programs to read, update and/or delete the data from the specific member overridden. 

From RPGLE Program

To work with multi-member physical files in RPGLE, We can use the EXTMBR keyword against the file definition in F-Spec. 

This would open the specific member of the file for processing (i.e., read, write, update and/or delete) the data from the specific member through out the program.

FMYFILE     UF   E             K DISK   EXTMBR('MEMBER1')

These operations can be performed by using the RPGLE Op-codes READ, WRITE, UPDATE and/or DELETE just by specifying the file name next to the op-codes and no need to specify the member name every time. 

If we need to open multiple members of the file in the same program, there are couple of ways based on how we need to access the data. 
  • By using the variable with EXTMBR keyword and defining the file in USROPN.
    • This would require the member name to be populated in the variable and open the file. 
    • When the processing with the member is completed, close the file, populate the next member and open the file. 
    • If data from all the members is to be read, *ALL can be used with EXTMBR.
    • If the member names are unknown, we could probably use DSPFD to dump the member list to OUTFILE and read the file in the program, use the member name from the OUTFILE in EXTMBR and open the file. 
  • If we need to access the data from two different members at the same time.
    • Define files with two different names in F-spec with EXTFILE pointing to the same file.
    • Using EXTMBR to specify the corresponding member name. 
    • Renaming the record format using RENAME keyword (as all the members of the PF would have same record format). 
    • Prefix the field names using PREFIX keyword to identify the field names belonging to the corresponding member in the program. 
Another way is to override the file in CL program and declare the file directly. 

From SQL

Be it an interactive SQL session or Embedded SQL query in SQLRPGLE, SELECT doesn't allow defining the member name. 

If we need to query the specific member of the file, we could create an alias and query on the alias instead. 

CREATE ALIAS QTEMP/ALIAS1 FOR QTEMP/MYFILE (MEMBER1)

This alias can be used to run SELECT, INSERT, UPDATE and/or DELETE queries. 

SELECT * FROM QTEMP/ALIAS1

I hope this have given you an insight on how to work with multi-member physical files in IBM i.


If you have any Suggestions or Feedback, Please leave a comment below or use Contact Form.

Different Ways of Sorting Data in a List - Python

Sorting Data in a List List is a collection of data (of different data types), much like an array. Like any data structure or data set, dat...