List

List is a container data type, similar to tuple, with lots of added functionality and mutable. Lists are typically used to store and manipulate ordered collection of values.

info Tuple and Sequence operations chapter is a significant prerequisite for this one.

Initialization and Slicing

Lists are declared as a comma separated values within [] square brackets. Unlike tuple there's no ambiguity in using [] characters, so there's no special requirement of trailing comma for a single element list object. You can use a trailing comma if you wish, which is helpful to easily change a list declared across multiple lines.

# 1D example
>>> vowels = ['a', 'e', 'i', 'o', 'u']
>>> vowels[0]
'a'
# same as vowels[4] since len(vowels) - 1 = 4
>>> vowels[-1]
'u'

# 2D example
>>> student = ['learnbyexample', 2021, ['Linux', 'Vim', 'Python']]
>>> student[1]
2021
>>> student[2]
['Linux', 'Vim', 'Python']
>>> student[2][-1]
'Python'

Since list is a mutable data type, you can modify the object after initialization. You can either change a single element or use slicing notation to modify multiple elements.

>>> nums = [1, 4, 6, 22, 3, 5]

>>> nums[0] = 100
>>> nums
[100, 4, 6, 22, 3, 5]

>>> nums[-3:] = [-1, -2, -3]
>>> nums
[100, 4, 6, -1, -2, -3]

# list will automatically shrink/expand as needed
>>> nums[1:4] = [2000]
>>> nums
[100, 2000, -2, -3]
>>> nums[1:2] = [3.14, 4.13, 6.78]
>>> nums
[100, 3.14, 4.13, 6.78, -2, -3]

List methods and operations

This section will discuss some of the list methods and operations. See docs.python: list methods for documentation. As mentioned earlier, you can use dir(list) to view the available methods of an object.

Use the append() method to add a single element to the end of a list object. If you need to append multiple items, you can pass an iterable to the extend() method. As an exercise, check what happens if you pass an iterable to the append() method and a non-iterable value to the extend() method. What happens if you pass multiple values to both these methods?

>>> books = []
>>> books.append('Cradle')
>>> books.append('Mistborn')
>>> books
['Cradle', 'Mistborn']

>>> items = [3, 'apple', 100.23]
>>> items.extend([4, 'mango'])
>>> items
[3, 'apple', 100.23, 4, 'mango']
>>> items.extend((-1, -2))
>>> items.extend(range(3))
>>> items.extend('hi')
>>> items
[3, 'apple', 100.23, 4, 'mango', -1, -2, 0, 1, 2, 'h', 'i']

The count() method will give the number of times a value is present.

>>> nums = [1, 4, 6, 22, 3, 5, 2, 1, 51, 3, 1]
>>> nums.count(3)
2
>>> nums.count(31)
0

The index() method will give the index of the first occurrence of a value. As seen with tuple, this method will raise ValueError if the value isn't present.

>>> nums = [1, 4, 6, 22, 3, 5, 2, 1, 51, 3, 1]
>>> nums.index(3)
4

The pop() method removes the last element of a list by default. You can pass an index to delete that specific item and the list will be automatically re-arranged. Return value is the element being deleted.

>>> primes = [2, 3, 5, 7, 11]
>>> last = primes.pop()
>>> last
11
>>> primes
[2, 3, 5, 7]
>>> primes.pop(2)
5
>>> primes
[2, 3, 7]

>>> student = ['learnbyexample', 2021, ['Linux', 'Vim', 'Python']]
>>> student.pop(1)
2021
>>> student[-1].pop(1)
'Vim'
>>> student
['learnbyexample', ['Linux', 'Python']]
>>> student.pop()
['Linux', 'Python']
>>> student
['learnbyexample']

To remove multiple elements using slicing notation, use the del statement. Unlike the pop() method, there is no return value.

>>> nums = [1.2, -0.2, 0, 2, 4, 23]
>>> del nums[0]
>>> nums
[-0.2, 0, 2, 4, 23]
>>> del nums[2:4]
>>> nums
[-0.2, 0, 23]

>>> nums_2d = [[1, 3, 2, 10], [1.2, -0.2, 0, 2], [100, 200]]
>>> del nums_2d[0][1:3]
>>> del nums_2d[1]
>>> nums_2d
[[1, 10], [100, 200]]

The pop() method deletes an element based on its index. Use the remove() method to delete an element based on its value. You'll get ValueError if the value isn't found.

>>> even_numbers = [2, 4, 6, 8, 10]
>>> even_numbers.remove(8)
>>> even_numbers
[2, 4, 6, 10]

The clear() method removes all the elements. You might wonder why not just assign an empty list? If you have observed closely, all of the methods seen so far modified the list object in-place. This is useful if you are passing a list object to a function and expect the function to modify the object itself instead of returning a new object. See Mutability chapter for more details.

>>> nums = [1.2, -0.2, 0, 2, 4, 23]
>>> nums.clear()
>>> nums
[]

You've already seen how to add element(s) at the end of a list using append() and extend() methods. The insert() method is the opposite of pop() method. You can provide a value to be inserted at the given index. As an exercise, check what happens if you pass a list value. Also, what happens if you pass more than one value?

>>> books = ['Sourdough', 'Sherlock Holmes', 'To Kill a Mocking Bird']
>>> books.insert(2, 'The Martian')
>>> books
['Sourdough', 'Sherlock Holmes', 'The Martian', 'To Kill a Mocking Bird']

The reverse() method reverses a list object in-place. Use slicing notation if you want a new object.

>>> primes = [2, 3, 5, 7, 11]
>>> primes.reverse()
>>> primes
[11, 7, 5, 3, 2]

>>> primes[::-1]
[2, 3, 5, 7, 11]
>>> primes
[11, 7, 5, 3, 2]

Here's some examples with comparison operators. Quoting from documentation:

For two collections to compare equal, they must be of the same type, have the same length, and each pair of corresponding elements must compare equal (for example, [1,2] == (1,2) is false because the type is not the same).

Collections that support order comparison are ordered the same as their first unequal elements (for example, [1,2,x] <= [1,2,y] has the same value as x <= y). If a corresponding element does not exist, the shorter collection is ordered first (for example, [1,2] < [1,2,3] is true).

>>> primes = [2, 3, 5, 7, 11]
>>> nums = [2, 3, 5, 11, 7]
>>> primes == nums
False
>>> primes == [2, 3, 5, 7, 11]
True

>>> [1, 1000] < [2, 3]
True
>>> [1000, 2] < [1, 2, 3]
False

>>> ['a', 'z'] > ['a', 'x']
True
>>> [1, 2, 3] > [10, 2]
False
>>> [1, 2, 3] > [1, 2]
True

Sorting and company

The sort() method will order the list object in-place. The sorted() built-in function provides the same functionality for iterable types and returns an ordered list.

>>> nums = [1, 5.3, 321, 0, 1, 2]

# ascending order
>>> nums.sort()
>>> nums
[0, 1, 1, 2, 5.3, 321]

# descending order
>>> nums.sort(reverse=True)
>>> nums
[321, 5.3, 2, 1, 1, 0]

>>> sorted('fuliginous')
['f', 'g', 'i', 'i', 'l', 'n', 'o', 's', 'u', 'u']

The key argument accepts the name of a built-in/user-defined function (i.e. function object) for custom sorting. If two elements are deemed equal based on the result of the function, the original order will be maintained (known as stable sorting). Here's some examples:

# based on the absolute value of an element
# note that the input order is maintained for all three values of "4"
>>> sorted([-1, -4, 309, 4.0, 34, 0.2, 4], key=abs)
[0.2, -1, -4, 4.0, 4, 34, 309]

# based on the length of an element
>>> words = ('morello', 'irk', 'fuliginous', 'crusado', 'seam')
>>> sorted(words, key=len, reverse=True)
['fuliginous', 'morello', 'crusado', 'seam', 'irk']

If the custom user-defined function required is just a single expression, you can create anonymous functions with lambda expressions instead of a full-fledged function. As an exercise, read docs.python HOWTOs: Sorting and implement the below examples using operator module instead of lambda expressions.

# based on second element of each item
>>> items = [('bus', 10), ('car', 20), ('jeep', 3), ('cycle', 5)]
>>> sorted(items, key=lambda e: e[1], reverse=True)
[('car', 20), ('bus', 10), ('cycle', 5), ('jeep', 3)]

# based on number of words, assuming space as the word separator
>>> dishes = ('Poha', 'Aloo tikki', 'Baati', 'Khichdi', 'Makki roti')
>>> sorted(dishes, key=lambda s: s.count(' '), reverse=True)
['Aloo tikki', 'Makki roti', 'Poha', 'Baati', 'Khichdi']

You can use sequence types like list or tuple to specify multiple sorting conditions. Make sure to read the sequence comparison examples from previous section before trying to understand the following examples.

>>> dishes = ('Poha', 'Aloo tikki', 'Baati', 'Khichdi', 'Makki roti')

# word-count and dish-names, both descending order
>>> sorted(dishes, key=lambda s: (s.count(' '), s), reverse=True)
['Makki roti', 'Aloo tikki', 'Poha', 'Khichdi', 'Baati']

# word-count descending order, dish-names ascending order
# the main trick is to negate the numerical value
>>> sorted(dishes, key=lambda s: (-s.count(' '), s))
['Aloo tikki', 'Makki roti', 'Baati', 'Khichdi', 'Poha']

As an exercise, given nums = [1, 4, 5, 2, 51, 3, 6, 22], determine and implement the sorting condition based on the required output shown below:

  • [4, 2, 6, 22, 1, 5, 51, 3]
  • [2, 4, 6, 22, 1, 3, 5, 51]
  • [22, 6, 4, 2, 51, 5, 3, 1]

Here's some examples with min() and max() functions.

>>> nums = [321, 0.5, 899.232, 5.3, 2, 1, -1]
>>> min(nums)
-1
>>> max(nums)
899.232
>>> min(nums, key=abs)
0.5

Random items

You have already seen a few examples with random module in earlier chapters. This section will show a few examples with methods that act on sequence data types.

First up, getting a random element from a non-empty sequence using the choice() method.

>>> import random

>>> random.choice([4, 5, 2, 76])
76
>>> random.choice('hello')
'e'

The shuffle() method randomizes the elements of a list in-place.

>>> items = ['car', 20, 3, 'jeep', -3.14, 'hi']

>>> random.shuffle(items)
>>> items
['car', 3, -3.14, 'jeep', 'hi', 20]

Use the sample() method to get a list of specified number of random elements. As an exercise, see what happens if you pass a slice size greater than the number of elements present in the input sequence.

>>> random.sample((4, 5, 2, 76), k=3)
[4, 76, 2]

>>> random.sample(range(1000), k=5)
[490, 26, 9, 745, 919]

Map, Filter and Reduce

Many operations on container objects can be defined in terms of these three concepts. For example, if you want to sum the square of all even numbers:

  • separating out even numbers is Filter (i.e. only elements that satisfy a condition are retained)
  • square of such numbers is Map (i.e. each element is transformed by a mapping function)
  • final sum is Reduce (i.e. you get one value out of multiple values)

One or more of these operations may be absent depending on the problem statement. A function for the first of these steps could look like:

>>> def get_evens(iterable):
...     op = []
...     for n in iterable:
...         if n % 2 == 0:
...             op.append(n)
...     return op
... 
>>> get_evens([100, 53, 32, 0, 11, 5, 2])
[100, 32, 0, 2]

Function after the second step could be:

>>> def sqr_evens(iterable):
...     op = []
...     for n in iterable:
...         if n % 2 == 0:
...             op.append(n * n)
...     return op
... 
>>> sqr_evens([100, 53, 32, 0, 11, 5, 2])
[10000, 1024, 0, 4]

And finally, the function after the third step could be:

>>> def sum_sqr_evens(iterable):
...     total = 0
...     for n in iterable:
...         if n % 2 == 0:
...             total += n * n
...     return total
... 
>>> sum_sqr_evens([100, 53, 32, 0, 11, 5, 2])
11028

info Python also provides map(), filter() and functools.reduce() for such problems. But, see Comprehensions and Generator expressions chapter before deciding to use them.

Here's some examples with sum(), all() and any() built-in reduce functions.

>>> sum([321, 0.5, 899.232, 5.3, 2, 1, -1])
1228.032

>>> conditions = [True, False, True]
>>> all(conditions)
False
>>> any(conditions)
True
>>> conditions[1] = True
>>> all(conditions)
True

>>> nums = [321, 1, 1, 0, 5.3, 2]
>>> all(nums)
False
>>> any(nums)
True

Exercises

  • Write a function that returns the product of a sequence of numbers. Empty sequence or sequence containing non-numerical values should raise TypeError.

    • product([-4, 2.3e12, 77.23, 982, 0b101]) should give -3.48863356e+18
    • product(range(2, 6)) should give 120
    • product(()) and product(['a', 'b']) should raise TypeError

  • Write a function that removes dunder names from dir() output.

    >>> remove_dunder(list)
['append', 'clear', 'copy', 'count', 'extend', 'index',
 'insert', 'pop', 'remove', 'reverse', 'sort']
>>> remove_dunder(tuple)
['count', 'index']