Mastodon Icon RSS Icon GitHub Icon LinkedIn Icon RSS Icon

How to use Rust in Python (Part 2)

You can find the first part of this article HERE.

In the previous part we have seen how to run simple Rust functions with integer arguments. This is not enough, of course. We need to go further by passing Python lists to Rust functions.

The problem is that it is not possible to pass directly a Python list to a C interface. Python lists (we can call them Plists) are complicated beasts, you can easily see that they are objects full of methods, and attributes and… Stuff.

1
2
dir([])
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

We need first to convert this in something edible from a Rust library. But first things first.

The Rust Function: Sum a List of Integers

The first step to do is to write the Rust function in our library. This time we want a function that, given a list of integers as input, returns the sum of the list. The final code is something like this:

1
2
3
4
5
#[no_mangle]
pub extern fn sum_list(data: *const int32_t, length: size_t) -> int32_t {
    let nums = unsafe { slice::from_raw_parts(data, length as usize) };
    nums.iter().fold(0, |acc, i| acc + i)
}

It is not a complicated function, but you can see that there are some interesting points:

  1. The function takes two arguments: a pointer to an int32_t which represent the first item on our list; and a size_t value representing the size of the list. We cannot pass a list directly, but we need to pass the list in its “primordial form”: an array of data in memory and the length of that array.
  2. Because the list is given as raw data, we need to “assemble” the list according the Rust fashion. This is what I’m doing in the unsafe block: create a slice from the raw parts.

Then the function continues as usual, folding the list in order to get the final sum. (Remember that no semi-colon at the end of a statement is just like a “return”.

The Python Side

It is time to go to the Python side of the application. Let’s write something that uses our brand new sum-list function.

1
2
3
4
5
6
lib.sum_list.argtypes = (ctypes.POINTER(ctypes.c_int32), ctypes.c_size_t)
print("Summing in Rust the list of first 1000 numbers.")
number_list = list(range(1001))
c_number_list = (ctypes.c_int32 * len(number_list))(*number_list)
result = lib.sum_list(c_number_list, len(number_list))
print("Result is {}. Expected 500500.".format(result))

There is a bit of magic there that I need to explain.

  1. First line. This explicitly defines the type of the arguments of sum_list. To be honest the script works even without this line, but I think it is better to use it anyway.
  2. c_number_list is where we explode the pythonic list into a C friendly list. It is a bit cryptic, but in short this mean: create a new Python object representing an array of len(number_list) blocks of c_int32; then initialize this object with the contents  of the desired list (number_list).
  3. Then we simply call the Rust function with this object (representing the raw data) and the length (given by the length of the list).

That’s it! :)

A nice side effect. Because we are using just the raw content of the list by “unpacking” the list (the asterisk before number_list on line 4), the same code works even if number_list is a tuple or a set.

In the next part

Cool! Now we are able to pass a Python list to a Rust function. But we can do much, much better! **In the next part we will see how to pass custom Python object to a Rust function! **

Imagine the possibilities! :)