Skip to content

Chapter 1 - Lists and dictionaries

Goal

  • Get comfortable with Python lists and dictionaries
  • Learn to manipulate them to your will without having to google for common operations
  • Practice till it becomes second nature

Introduction

As a hardware engineer, most of the Python code I write centers around two data structures

  1. Python List [] - which can be compared to SystemVerilog Queues
  2. Python Dictionary {} - which is equivalent to SystemVerilog Associative Arrays

Practice

Try out the examples in this tutorial either by launching python3 on your Linux or MacOS terminal, or using this Python web interpreter.

Python list

Interacting with lists

In any programming language (C, SystemVerilog, Perl, etc), we typically interact with arrays in a few common ways

  • Store stuff in them
  • Push and pop items from them
  • Slice arrays to select a portion of it, or combine arrays to make a larger one
  • Check how many items are there in it

Here are some of these actions in Python compared against SystemVerilog queues.

Action SystemVerilog Python Comment
initialize int k[$] = {1, 2, 3} k = [1, 2, 3]
access element k[0], k[1] k[0], k[1]
k[-1], k[-2]		 In Python you can access elements from the end using negative indices
push, insert k.push_front(100)
k.push_back(101)
k.insert(2, 20)		 k.insert(0, 100)
k.insert(-1, 101)
k.append(101)		 Notice how you can address the last index using -1
pop k.pop_front()
k.pop_back()		 k.pop(0)
k.pop(-1)
size k.size() len(k)
slicing k[0:3]
k[4:$]		 k[:3]
k[4:]		 Omitting first element starts index from 0
Omitting second index indicates end of list
clear k.delete() k.clear()
copy m = k m = k[:]
m = k.copy()
m = list(k)		 Three different ways to copy lists
exists Iterate and check using
foreach k[i]		 if 2 in k:
  return True		 (<elem> in <list>) will return True
if element exists, else False

More things you can do with lists

Instead of having to iterate over a list in a for loop, Python offers convenient functions to perform the following actions.

  • reverse
  • sum
  • find min and max elements
  • sort
  • concatenate lists
  • count occurence
  • join
# Reverse a list in place. This modifies the list.
>>> k = [1, 2, 3, 4]
>>> k.reverse() 
>>> k
[4, 3, 2, 1]

# Sum all elements of the list
>>> sum([10, 20, 30, 40, 50])
150

# Sum all elements but add a base value to it
>>> sum([10, 20, 30, 40, 50], start=1000)
1150

# Finding min and max elements
>>> min([10, 20, 30, 40, 50])
10

>>> max([10, 20, 30, 40, 50])
50

# Concatenate lists
>>> k.extend([5, 6, 7])
>>> k
[1, 2, 3, 4, 5, 6, 7]

>>> m = [10, 11, 12]
>>> k + m
[1, 2, 3, 4, 5, 6, 7, 10, 11, 12]

# Sort list
>>> k.sort() #ascending
>>> k.sort(reverse=True) #descending

# Join
'/'.join(['apple', 'banana', 'pear'])
'+'.join(['apple', 'banana', 'pear'])
' '.join(['apple', 'banana', 'pear'])

Python Dictionary

Interacting with dicts

Dictionaries (dict for short), similar to SV associative arrays, are used to store [key][value] pairs. Here are some common ways to interact with dicts.

# Initialize
>>> d = {'apple': 'fruit', 'horse': 'animal', 'orange': 'fruit', 'dog': 'animal'}

# Insert
>>> d['mango'] = 'fruit'

# Update
# I'm changing the value of 'dog' and adding 2 new keys
>>> d.update({'dog': 'pet', 'banana': 'fruit', 'kiwi': 'fruit'})

# Delete
>>> del d['orange']

# Check if key exists
>>> 'dog' in d
True
>>> 'cat' in d
False

Using the get method

In Python, you have two ways to fetch an entry from a dict

  • Simply access it d['orange']
  • Or retrieve using the .get() method

If you access a key that does not exists (such as k['cat'] in our example), a KeyError exception is raised. This can sometimes be problematic, because you don't want your script to error out. Instead, using k.get('cat') to retrieve an entry will cleanly return None and not cause errors.

.get default

Also, dict.get(<key>, <default>) takes an optional second argument. Instead of having .get() return None, you can make it return a different value if a key does not exist.

dict.get(<key>, <default>)

>>> d['cat']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'cat'

# No exception thrown
>>> d.get('cat')

# Get a different default value instead of None, 
# in case the key does not exist
>>> d.get('cat', 'animal')
animal

Iterating over dicts

Iterating over dicts is a little different from lists

  • Simply doing for item in d: will only fetch the keys
  • for key, value in d.items() will fetch both, key and value, for each entry in the dict.
>>> d = {'apple': 'fruit', 'horse': 'animal', 'orange': 'fruit', 'dog': 'animal'}
>>> for item in d:
...   print(item)
... 
apple
horse
orange
dog

>>> for k, v in d.items():
...   print(k, '=>', v)
... 
apple => fruit
horse => animal
orange => fruit
dog => animal

Making copies

Shallow copy

  • In SV you can copy a queue by making a simple assignment m=k.
  • But in Python this assignment m=k will make m point to the same memory location as k, so modifying m will also modify k 
# Wrong way to copy a list
>>> k = [1, 2, 3]
>>> m = k
>>> m.append(4)
>>> print(k)
[1, 2, 3, 4]

# Correct way to copy a list
>>> m = k[:]
>>> m = k.copy()
>>> m = list(k)

# Wrong way to copy a dictionary
>>> d = {'name': 'john', 'age': '40'}
>>> s = d
>>> s['age'] = 100
>>> d
{'name': 'john', 'age': 100}

# Correct way to copy a dictionary
>>> s = d.copy()
>>> d = dict(d)

Deep Copy

  • If you have a list within a list such as a 2D array, or a list within a dict, then a simple .copy() will only perform a shallow copy.
  • To clone such a multi-dimensional structure, you will have to use copy.deepcopy()
  • Here we access the deepcopy() utility present in the copy library by doing a import copy.
  • The import copy statement is similar to import pkg; in SV or #include <stdio.h> in C. (More on this in a later chapter).
  • In this example, lets consider a 2D matrix k and a dict d with a list inside it and examine how they are affected by shallow copy versus deep copy.
# 2D matrix
k = [[1, 2], [3, 4]]

# List within a dict
d = {'name': 'john', 'num': [1, 2, 3]}
# Shallow copy of a 2D list
>>> m = k[:]
>>> m[0][0] = 1000
>>> k # changing `m` also changes `k`

# Deep copy of a 2D list
>>> import copy
>>> m = copy.deepcopy(k)
>>> m[0][0] = 1000
>>> k # changing `m` does NOT change `k`

# Shallow copy of dict
>>> s = d.copy()
>>> s['num'][0] = 1000
>>> d # changing `s` also changes `d`

# Deep copy of dict
>>> import copy
>>> s = copy.deepcopy(d)
>>> s['num'][0] = 1000
>>> d # changing `s` does NOT change `d`

List and Dictionary Comprehension

One of the actions tht I perform frequently is iterating over lists and dicts to create new lists or dicts. The obvious way to do this is using a for loop. But, Python offers a more convenient and terse way to do the same. Here are some examples.

# Given a list of numbers add 100 to each of them
>>> k = [1, 2, 3, 4, 5]
>>> m = [i+100 for i in k]
>>> m
[101, 102, 103, 104, 105]

# Filter out names that start with the letter `m`
>>> names = ['tony', 'mike', 'jack', 'bryan', 'mark', 'matthew', 'nancy', 'miles']
>>> m = [n for n in names if n[0] == 'm']
>>> m
['mike', 'mark', 'matthew', 'miles']
# Add 100 to values of each key in the dict
>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> m = {k:v+100 for k,v in d.items()}
>>> m
{'a': 101, 'b': 102, 'c': 103}

# Add a suffix `_name` to all the keys in the dict
>>> m = {k+'_name':v for k,v in d.items()}
>>> m
{'a_name': 1, 'b_name': 2, 'c_name': 3}

List/Dict Comprehension

If you find yourself writing for loops to iterate over lists or dict, try to see if you can refactor your code and rewrite it using list/dict comprehensions

Practice, Practice, Practice

  • The key to learning Python is practice
  • Set 15 minutes on a timer, fire up python3 on your Linux or MacOS terminal (or use this Python web interpreter) and practice some of the code in this chapter.

Back to Table of Contents

Table of contents