Module 0 - Installing the tools

Slides

In this file you'll find instructions on how to install the tools we'll use during the workshop.

All of these tools are available for Linux, macOS and Windows users. We'll need the tools to write and compile our Rust code. Important: these instructions are to be followed at home, before the start of the first workshop. If you have any problems with the installation process, please contact us!

Rust and Cargo

First we'll need rustc, the standard Rust compiler. rustc is generally not invoked directly, but through cargo, the Rust package manager. rustup takes care of installing rustc and cargo.

This part is easy: go to https://rustup.rs and follow the instructions. Please make sure you're installing the latest default toolchain. Once done, run

rustc -V && cargo -V

The output should be something like this:

rustc 1.68.2 (9eb3afe9e 2023-03-27)
cargo 1.68.2 (6feb7c9cf 2023-03-26)

Using Rustup, you can install Rust toolchains and components. More info:

• https://rust-lang.github.io/rustup
• https://doc.rust-lang.org/cargo

Rustfmt and Clippy

To avoid discussions, Rust provides its own formatting tool, Rustfmt. We'll also be using Clippy, a collection of lints to analyze your code, that catches common mistakes for you. You'll notice that Rusts Clippy can be a very helpful companion. Both Rustfmt and Clippy are installed by Rustup by default.

To run Rustfmt on your project, execute:

cargo fmt

To run clippy:

cargo clippy

More info:

• Rustfmt: https://github.com/rust-lang/rustfmt
• Clippy: https://github.com/rust-lang/rust-clippy

Visual Studio Code

During the workshop, you can use Visual Studio Code (vscode) to write code in. Of course, you're free to use your favorite editor, but if you encounter problems, we can't be of very much help.

You can find the installation instructions here: https://code.visualstudio.com/.

We will install the Rust-Analyzer plugin as well. The first one is Rust-Analyzer. Installation instructions can be found here https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer. Rust-Analyzer provides a lot of help during development and in indispensable when getting started with Rust.

More info:

• https://rust-analyzer.github.io/

Git

We will use Git as version control tool. If you haven't installed Git already, you can find instructions here: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git. If you're new to Git, you'll also appreciate GitHubs intro to Git https://docs.github.com/en/get-started/using-git/about-git and the Git intro with vscode, which you can find here: https://www.youtube.com/watch?v=i_23KUAEtUM.

More info: https://www.youtube.com/playlist?list=PLg7s6cbtAD15G8lNyoaYDuKZSKyJrgwB-

Workshop code

Now that everything is installed, you can clone the source code repository. The repository can be found here: https://github.com/tweedegolf/rust-workshop.

Clone the repository. Instructions on cloning the repository can be found here: https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls

Trying it out

Now that you've got the code on your machine, navigate to it using your favorite terminal and run:

cd exercises/0-intro/host
cargo run

This command may take a while to run the first time, as Cargo will first fetch the crate index from the registry. It will compile and run the intro package, which you can find in exercises/0-intro/host. If everything goes well, you should see some output:

   Compiling intro v0.1.0 ([REDACTED]/rust-workshop/exercises/0-intro/host)
    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/intro`
🦀 Hello, world! 🦀
You've successfully compiled and run your first Rust project!

If Rust-Analyzer is set up correctly, you can also click the '▶️ Run'-button that is shown in exercises/0-intro/host/src/main.rs.

Instructions for embedded

This part is relevant only if you're partaking in one of the workshops on embedded Rust.

Hardware

You should have received the following parts:

• nRF52840-DK
• Breadboard
• LIS3DH Breakout board
• Male-to-male breadboard wires

You'll also need a Micro-USB cable, but we're sure you've got one to spare.

Please check that everything is complete. If not, please contact us.

Software

Then, we'll install some tools needed to flash the mcu and inspect the code.

Install the thumbv7em-none-eabihf toolchain with the following command:

rustup target add thumbv7em-none-eabihf

On linux you need to install the "dev" libraries for udev, usb, and ftdi libudev-dev. If you're on Ubuntu:

# ubuntu
sudo apt install -y libusb-1.0-0-dev libftdi1-dev libudev-dev

On all platforms:

rustup component add llvm-tools-preview rustfmt clippy
cargo install probe-run

If you're on linux, you'll need to update your udev rules. On ubuntu, run the following inside the workshop folder you just cloned;

sudo cp 99-jlink-nrf.rules /etc/udev/rules.d
sudo udevadm control --reload-rules

If you're on windows, we need to install a generic WinUSB driver. You can use Zadig to select the usb device that uses the jlink driver and install WinUSB on it. This will uninstall the official driver, which means that the official Segger tools will not work anymore after this. To revert, go to device manager and uninstall the usb device. The jlink driver will then be used again for that usb connection.

Then, switch the DK off and on or remove the cable and plug it in again.

Trying it out

Before we begin, we need to test our hardware. We'll be testing the LIS3DH accelerometer, as well as the nRF52840-DK board. Make sure you have checked out the latest version of the workshop source.

LIS3DH accelerometer connection

First, let's wire up the LIS3DH accelerometer for I2C. Please turn off your DK. Then, wire up the accelerometer, referring to the table below.

We'll be using other pins later on, but they're not needed to test the hardware

Running the test

To test the hardware, please connect the nRF52840-DK to your pc, switch it on, and run

cd ./exercises/0-intro/embedded
cargo run --release --bin test

If everything works correctly, you should now see the accelerometer samples being printed on the display. If not, don't worry and contact us.

If not, you may have an accelerometer that uses the alternate i2c address. If so, run this instead:

cargo run --release --bin test --features alternate-addr

Docs

Datasheets, manuals, and schematics of the parts we are using in the embedded workshops.

nRF52840

• nRF52840DK documentation
• nRF52840 product specification

LIS3DH

• Datsheet
• Schematic (Adafruit)
• Schematic (SparkFun)

Module A1 - Basic Syntax

Slides

A1.1 Basic syntax

Open exercises/A1/1-basic-syntax in your editor. This folder contains a number of exercises with which you can practise basic Rust syntax.

While inside the exercises/A1/1-basic-syntax folder, to get started, run:

cargo run --bin 01

This will try to compile exercise 1. Try and get the example to run, and continue on with the next exercise by replacing the number of the exercise in the cargo run command.

Some exercises contain unit tests. To run the test in src/bin/01.rs, run

cargo test --bin 01

Make sure all tests pass!

Module A2 - Ownership & references

Slides

A2.1 Move semantics

This exercise is adapted from the move semantics exercise from Rustlings

This exercise enables you to practise with move semantics. It works similarly to exercise A1.1. To get started, exercises/A2/1-move-semantics in your editor and run

cargo run --bin 01

Make all exercises compile. For some of them, extra instructions are included as doc comments at the top of the file. Make sure to adhere to them.

A2.2 Borrowing

Fix the two examples in the exercises/A2/2-borrowing crate! Don't forget you can run individual binaries by using cargo run --bin 01 in that directory! Make sure to follow the instructions that are in the comments!

Module A3 Advanced Syntax

Slides

A3.1 Error Propagation

Follow the instructions in the comments of excercises/A3/1-error-propagating/src/main.rs!

A3.2 Slices

Follow the instructions in the comments of excercises/A3/2-slices/src/main.rs! Don't take too much time on the extra assignment, instead come back later once you've done the rest of the excercises.

A3.3 Error Handling

Follow the instructions in the comments of excercises/A3/3-error-handling/src/main.rs!

A3.4 Boxed Data

Follow the instructions in the comments of excercises/A3/4-boxed-data/src/main.rs!

A3.5 Bonus - Ring Buffer

This is a bonus exercise! Follow the instructions in the comments of excercises/A3/5-bonus-ring-buffer/src/main.rs!

Module A4 - Traits and generics

Slides

A4.1 Local Storage Vec

In this exercise, we'll create a type called LocalStorageVec, which is generic list of items that resides either on the stack or the heap, depending on its size. If its size is small enough for items to be put on the stack, the LocalStorageVec buffer is backed by an array. LocalStorageVec is not only generic over the type (T) of items in the list, but also by the size (N) of this stack-located array using a relatively new feature called 'const generics'. Once the LocalStorageVec contains more items than fit in the array, a heap based Vec is allocated as space for the items to reside in.

Questions

1. When is such a data structure more efficient than a standard Vec?
2. What are the downsides, compared to just using a Vec?

Open the exercises/A4/local-storage-vec crate. It contains a src/lib.rs file, meaning this crate is a library. lib.rs contains a number of tests, which can be run by calling cargo test. Don't worry if they don't pass or even compile right now: it's your job to fix that in this exercise. Most of the tests are commented out right now, to enable a step-by-step approach. Before you begin, have a look at the code and the comments in there, they contain various helpful clues.

A4.1 A Defining the type

Currently, the LocalStorageVec enum is incomplete. Give it two variants: Stack and Heap. Stack contains two named fields, buf and len. buf will be the array with a capacity to hold N items of type T; len is a field of type usize that will denote the amount of items actually stored. The Heap variant has an unnamed field containing a Vec<T>. If you've defined the LocalStorageVec variants correctly, running cargo test should output something like

running 1 test
test test::it_compiles ... ignored, This test is just to validate the definition of `LocalStorageVec`. If it compiles, all is OK

test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s

This test does (and should) not run, but is just there for checking your variant definition.

#![allow(unused)]
fn main() {
// Define an enum `LocalStorageVec` that is generic over
// type `T` and a constant `N` of type `usize`
pub enum LocalStorageVec<T, const N: usize> {
    // Define a struct-like variant called `Stack` containing two named fields:
    // - `buf` is an array with elements of `T` of size `N`
    // - `len` is a field of type `usize`
    Stack { buf: [T; N], len: usize },
    // Define a tuplle-like variant called `Heap`, containing a single field
    // of type `Vec<T>`, which is a heap-based growable, contiguous list of `T`
    Heap(Vec<T>),
}
}

A4.1 B impl-ing From<Vec<T>

Uncomment the test it_from_vecs, and add an implementation for From<Vec<T>> to LocalStorageVec<T>. To do so, copy the following code in your lib.rs file and replace the todo! macro invocation with your code that creates a heap-based LocalStorageVec containing the passed Vec<T>.

#![allow(unused)]
fn main() {
impl<T, const N: usize> From<Vec<T>> for LocalStorageVec<T, N> {
    fn from(v: Vec<T>) -> Self {
        todo!("Implement me");
    }
}
}

Question

1. How would you pronounce the first line of the code you just copied in English?*

Run cargo test to validate your implementation.

A4.1 C impl LocalStorageVec

To make the LocalStorageVec more useful, we'll add more methods to it. Create an impl-block for LocalStorageVec. Don't forget to declare and provide the generic paramereters. For now, to make implementations easier, we will add a bound T, requiring that it implements Copy and Default. First off, uncomment the test called it_constructs. Make it compile and pass by creating a associated function called new on LocalStorageVec that creates a new, empty LocalStorageVec instance without heap allocation.

The next methods we'll implement are len, push, pop, insert, remove and clear:

• len returns the length of the LocalStorageVec
• push appends an item to the end of the LocalStorageVec and increments its length. Possibly moves the contents to the heap if they no longer fit on the stack.
• pop removes an item from the end of the LocalStorageVec, optionally returns it and decrements its length. If the length is 0, pop returns None
• insert inserts an item at the given index and increments the length of the LocalStorageVec
• remove removes an item at the given index and returns it.
• clear resets the length of the LocalStorageVec to 0.

Uncomment the corresponding test cases and make them compile and pass. Be sure to have a look at the methods provided for slices [T] and Vec<T> Specifically, [T]::copy_within and Vec::extend_from_slice can be of use.

A4.1 D Iterator and IntoIterator

Our LocalStorageVec can be used in the real world now, but we still shouldn't be satisfied. There are various traits in the standard library that we can implement for our LocalStorageVec that would make users of our crate happy.

First off, we will implement the IntoIterator and Iterator traits. Go ahead and uncomment the it_iters test case. Let's define a new type:

#![allow(unused)]
fn main() {
pub struct LocalStorageVecIter<T, const N: usize> {
    vec: LocalStorageVec<T, N>,
    counter: usize,
}
}

This is the type we'll implement the Iterator trait on. You'll need to specify the item this Iterator implementation yields, as well as an implementation for Iterator::next, which yields the next item. You'll be able to make this easier by bounding T to Default when implementing the Iterator trait, as then you can use the std::mem::take function to take an item from the LocalStorageVec and replace it with the default value for T.

Take a look at the list of methods under the 'provided methods' section. In there, lots of useful methods that come free with the implementation of the Iterator trait are defined, and implemented in terms of the next method. Knowing in the back of your head what methods there are, greatly helps in improving your efficiency in programming with Rust. Which of the provided methods can you override in order to make the implementation of LocalStorageVecIter more efficient, given that we can access the fields and methods of LocalStorageVec?

Now to instantiate a LocalStorageVecIter, implement the [IntoIter] trait for it, in such a way that calling into_iter yields a LocalStorageVecIter.

A4.1 E AsRef and AsMut

AsRef and AsMut are used to implement cheap reference-to-reference coercion. For instance, our LocalStorageVec<T, N> is somewhat similar to a slice &[T], as both represent a contiguous series of T values. This is true whether the LocalStorageVec buffer resides on the stack or on the heap.

Uncomment the it_as_refs test case and implement AsRef<[T]> and AsMut<[T]>.

A4.1 F Index

To allow users of the LocalStorageVec to read items or slices from its buffer, we can implement the Index trait. This trait is generic over the type of the item used for indexing. In order to make our LocalStorageVec versatile, we should implement:

• Index<usize>, allowing us to get a single item by calling vec[1];
• Index<RangeTo<usize>>, allowing us to get the first n items (excluding item n) by calling vec[..n];
• Index<RangeFrom<usize>>, allowing us to get the last n items by calling vec[n..];
• Index<Range<usize>>, allowing us to get the items between n and m items (excluding item m) by calling vec[n..m];

Each of these implementations can be implemented in terms of the as_ref implementation, as slices [T] all support indexing by the previous types. That is, [T] also implements Index for those types. Uncomment the it_indexes test case and run cargo test in order to validate your implementation.

A4.1 G Removing bounds

When we implemented the borrowing Iterator, we saw that it's possible to define methods in separate impl blocks with different type bounds. Some of the functionality you wrote used the assumption that T is both Copy and Default. However, this means that each of those methods are only defined for LocalStorageVecs containing items of type T that in fact do implement Copy and Default, which is not ideal. How many methods can you rewrite having one or both of these bounds removed?

A4.1 H Borrowing Iterator

We've already got an iterator for LocalStorageVec, though it has the limitation that in order to construct it, the LocalStorageVec needs to be consumed. What if we only want to iterate over the items, and not consume them? We will need another iterator type, one that contains an immutable reference to the LocalStorageVec and that will thus need a lifetime annotation. Add a method called iter to LocalStorageVec that takes a shared &self reference, and instantiates the borrowing iterator. Implement the Iterator trait with the appropriate Item reference type for your borrowing iterator. To validate your code, uncomment and run the it_borrowing_iters test case.

Note that this time, the test won't compile if you require the items of LocalStorageVec be Copy! That means you'll have to define LocalStorageVec::iter in a new impl block that does not put this bound on T:

#![allow(unused)]
fn main() {
impl<T: Default + Copy, const N: usize> LocalStorageVec<T, N> {
    // Methods you've implemented so far
}

impl<T: const N: usize> LocalStorageVec<T, N> {
    pub fn iter(&self) -> /* TODO */
}
}

Defining methods in separate impl blocks means some methods are not available for certain instances of the generic type. In our case, the new method is only available for LocalStorageVecs containing items of type T that implement both Copy and Default, but iter is available for all LocalStorageVecs.

A4.1 I Generic Index

You've probably duplicated a lot of code in the last exercise. We can reduce the boilerplate by defining an empty trait:

#![allow(unused)]
fn main() {
trait LocalStorageVecIndex {}
}

First, implement this trait for usize, RangeTo<usize>, RangeFrom<usize>, and Range<usize>.

Next, replace the implementations from the previous exercise with a blanket implementation of Index. In English:

"For each type T, I and constant N of type usize, *implement Index<I> for LocalStorageVec<T, N>, where I implements LocalStorageVecIndex and [T] implements Index<I>"

If you've done this correctly, it_indexes should again compile and pass.

A4.1 J Deref and DerefMut

The next trait that makes our LocalStorageVec more flexible in use are Deref and DerefMut that utilize the 'deref coercion' feature of Rust to allow types to be treated as if they were some type they look like. That would allow us to use any method that is defined on [T] by calling them on a LocalStorageVec. Before continueing, read the section 'Treating a Type Like a Reference by Implementing the Deref Trait' from The Rust Programming Language (TRPL). Don't confuse deref coercion with any kind of inheritance! Using Deref and DerefMut for inheritance is frowned upon in Rust.

Below, an implementation of Deref and DerefMut is provided in terms of the AsRef and AsMut implementations. Notice the specific way in which as_ref and as_mut are called.

#![allow(unused)]
fn main() {
impl<T, const N: usize> Deref for LocalStorageVec<T, N> {
    type Target = [T];

    fn deref(&self) -> &Self::Target {
        <Self as AsRef<[T]>>::as_ref(self)
    }
}

impl<T, const N: usize> DerefMut for LocalStorageVec<T, N> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        <Self as AsMut<[T]>>::as_mut(self)
    }
}
}

Question

• Replacing the implementation of deref with self.as_ref() results in a stack overflow when running an unoptimized version. Why? (Hint: deref coercion)

Module B1: Embedded basics

Slides

This first part will give an overview of how the Rust Embedded ecosystem is built up.

We'll then use our newly gained knowledge to get our first application running and read out the ID of the LIS3DH accelerometer.

B1 LIS3DH ID

To get started we'll setup the i2c on our development kit and read out the ID register of the LIS3DH accelerometer. The starting point can be found in exercises/B1-B2/src/main of this repository.

Try to run the existing project and then fill in the functionality as instructed by the comments.

To use that project, you can use the following commands from inside that folder using the terminal:

• cargo build: Builds the project
• cargo run: Builds the project, flashes it to the device and listens for any logs which it will display in the terminal. (This uses the probe-run tool)

In both cases you can add the --release flag to turn on optimizations.

Note: There is a module called lis3dh in the assignment project. This is meant to be used in assignment B2, so it can be ignored for now.

Module B2: Cross-platform drivers

Slides

When you really want to use a device, you want to have a driver. We are going to learn about those next.

B2 LIS3DH Driver

Let's write an actual portable device driver for the accelerometer we've got.

Got to the assignment in ./exercises/B1-B2 and implement the lis3dh module. The goal is to use embedded-hal for our hardware definitions, so try not to use any nrf specific types in that module.

You should have all the information you need in the previous chapters of this book, but please do ask questions if you have any.

Module B3: RTIC

Slides

The goal of this exercise is to get acquainted with RTIC. Use your experience (and code) from parts B1 and B2.

B3 Simple RTIC app

LIS3DH accelerometer connection

First, let's wire up the LIS3DH accelerometer for I2C usage. We'll connect the LIS3DH INT1 pin to the nRF's P0.02. Please turn off your DK. Then, wire up the accelerometer, referring to the table below.

Instructions

1. For this exercise, we'll be working in exercises/B3. Inside the src folder, you'll find a couple of files:

   • lib.rs Where all modules are declared. No need to edit
   • hal_import.rs HAL compatibility module. You can leave it as is.
   • acc.rs contains a config_acc function. This function can be used to configure the LIS3DH to raise an interrupt if it experiences acceleration above 1.1g. It uses the lis3dh driver crate in order to do so. All you need to do is pass it a TWIM instance.
   • main.rs Here's where your magic happens. This is a typical RTIC application. It contains several examples of topics we just covered in the talk. You'll find the instructions at the bottom of the init task.

2. Follow the instructions in main.rs.

3. If you're done early, try to get some cool LED animation sequence going using task scheduling.

Resources

• The RTIC book
• nRF-HAL examples
• LIS3DH driver repository
• LIS3DH driver documentation

Module B4: Async on embedded

Slides

Using Embassy, we can run asynchronous Rust code on embedded devices. In this exercise, we'll give Embassy a try.

B4 Async LIS3DH ID

Just like in exercise B1, we will extract the ID register value from the LIS3DH. This time, though, we'll use Embassy to do this asyncronously. Open exercises/B4, examine the code in src/main.rs and run it.

This code requires the use of the nightly compiler. This should be downloaded automatically due to the rust-toolchain.toml file.

You can install and use the toolchain easily by running:

rustup toolchain install nightly
cargo +nightly build

If you don't want to provide the +nightly every time, we can make it the default:

rustup default nightly

You can find more embassy examples for the nrf52840 here: https://github.com/embassy-rs/embassy/tree/master/examples/nrf52840/src/bin

You can try getting the lis3dh to work in embassy with this driver: https://crates.io/crates/lis3dh-async

Module B5 - Rust for IoT

There are no slides for this module

In this module, we'll have a look at an example of a bigger Rust project.

B5 Device-host communication

The goal of this exercise is to get an idea of how code can be shared between a device and a host, in order to set up a robust communication system.

Instructions

1. For this exercice, we'll be working in exercises/B5. There are a couple of projects in there:

   • firmware contains all code that is run on the device. Apart from what you saw in the last exercise, it contains a uarte module, which uses the PAC to enable advanced functionality not implemented by the HAL. Take a peek at the functions in there. Try not to get distracted by the implementation details, but focus on the overall functionality that is available. The main application already implements tasks to control the TimeoutUarte.
   • cli defines a simple CLI application that listens for incoming messages, and opens a rudimentary repl with which you can send commands. You'll be implementing a couple of commands yourself, so have a peek at cmd.rs, to get an idea of how to do that.
   • format contains definitions of whatever is being send from the device to the server and vice-versa. To send new commands, you'll need to update the ServerToDevice and DeviceToServer structs. If you do, don't forget to compile both the firmware and the CLI in order for getting them to communicate nicely.

2. Flash the firmware onto the device using this command:

cd ./firmware
cargo run --release

5. Run the CLI app with one of these commands. For <PORT> substitute the device's serial port path. If you omit the argument, the app will print any serial ports detected.

cd ./cli
cargo run -- -p <PORT>

6. Test the setup. In the CLI repl, type

hello

Another command you can try (although it will only print stuff over RTT for now):

led 1 on

7. Open main.rs, and look up the handle_message task. In there, incoming messages are processed. You'll find the first instructions there.

8. Your next objective is to implement your own command. Add a command to cmd.rs in the cli package. Register it with the CommandParser::parse method, referring to the other commands in that module. Update the ServerToDevice and DeviceToServer structs in format, and handle the messages in firmware. If you need ideas, you can have the device send over accelerometer measurements whenever they're ready.

Resources

• The RTIC book
• Lis3dh driver documentation
• Lis3dh driver repository
• nRF-HAL examples

Module C1 - Parallel Rust

Slides

C.1 TF-IDF ★★

Follow the instructions in the comments of excercises/C1/1-tf-idf/src/main.rs!

C.2 Basic Mutex ★★★

Follow the instructions in the comments of excercises/C1/2-mutex/src/main.rs!

C.3 Advanced Mutex (bonus) ★★★★

The basic mutex performs a spin-loop while waiting to take the lock. That is terribly inefficient. Luckily, your operating system is able to wait until the lock becomes available, and will just put the thread to sleep in the meantime.

This functionality is exposed in the atomic_wait crate. The section on implementing a mutex from "Rust Atomics and Locks" explains how to use it.

• change the AtomicBool for a AtomicU32
• implement lock. Be careful about spurious wakes: after wait returns, you must stil check the condition
• implement unlocking (Drop for MutexGuard<T> using wake_one.

The linked chapter goes on to further optimize the mutex. This really is no longer part of this workshop, but we won't stop you if you try (and will still try to help if you get stuck)!

Module C2 - Async foundations

Slides

C2.1

We're going to build an observable variable, a bit similar in idea to a condvar.

It should have the following use:

#![allow(unused)]
fn main() {
pub static CPU_TEMPERATURE: Observable<f32> = Observable::new(20.0);

async fn throttle_if_cpu_temp_high(cpu: &mut Cpu) -> ! {
    loop {
        CPU_TEMPERATURE
            .wait_until(|temperature| temperature > 90.0)
            .await;

        cpu.throttle();

        CPU_TEMPERATURE
            .wait_until(|temperature| temperature < 80.0)
            .await;

        cpu.un_throttle();
    }
}
}

Go to exercise C2/1-observable and implement the type. Run cargo test in that folder to check if your implementation works.

Extra questions and challenges:

• For the embedded devs, make the library no_std
• Can we get rid of the Clone bound? If not, why not? If we can, what would the API look like?
• Async API design is hard. The Observable API has its limitations too. What are they? Can it be improved?
• Make the Observable support multiple wakers. (For no_std this is extra challenging)

Module C3 - Foreign Function Interface

Slides

This module is about having Rust code interact with C code and vice-versa.

Note: Bindgen depends on Clang being installed. See these instructions. This is required for the the 3rd exercise. For the first and exercise any C compiler will suffice, like clang, msvc and gcc.

C3.1 CRC in C

Use a CRC checksum function written in C in a Rust program

Steps

1. Add the cc build dependency, by adding to Cargo.toml the lines:

   [build-dependencies]
   cc = "1.0"
   

2. Create build.rs in the root of the project (next to Cargo.toml) with contents

   extern crate cc;
   
   fn main() {
       println!("cargo:rerun-if-changed=crc32.h");
       println!("cargo:rerun-if-changed=crc32.c");
       cc::Build::new().file("crc32.c").compile("crc32");
   }

   This will find your c code, compile it, and link it into the executable rust produces. It also instructs Cargo to re-compile the C code in case it changes.

3. In main.rs, define an extern (fill in the argument and return types)

   #![allow(unused)]
   fn main() {
   extern "C" {
       fn CRC32( ... ) -> ...; // hint: https://doc.rust-lang.org/std/os/raw
   }
   }

4. Now, create a rust wrapper that calls the extern function

   #![allow(unused)]
   fn main() {
   fn crc32( ... ) -> ... { 
       ... // (hints: `unsafe`, `.as_ptr()`, `.len()`)
   }
   }

5. Call our wrapper on some example input

   fn main() {
       println!("{:#x}", crc32(b"12345678"));
   }

   In the above example, the correct output is 0x9ae0daaf

C3.2 CRC in Rust

Use a CRC checksum function written in Rust in a C program

Steps

1. Change Cargo.toml to

   [package]
   name = "crc-in-rust"
   version = "0.1.0"
   edition = "2021"
   
   [lib]
   name = "crc_in_rust"
   crate-type = ["dylib"]
   
   # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
   
   [dependencies]
   

This instructs Cargo to compile our crate as a dynamic library.

2. Expose an extern rust function

   #![allow(unused)]
   fn main() {
   #[no_mangle]
   pub extern "C" fn crc32(...) -> ... {
   
       ...
   
       crc32_rust(...)
   }
   }

3. Create a C header file crc_in_rust.h

   #include <stdint.h> // uint32_t, uint8_t
   #include <stddef.h> // size_t
   
   uint32_t crc32(const uint8_t data[], size_t data_length);
   

4. Use the rust crc32 function in C

   #include <stdint.h> // uint32_t, uint8_t
   #include <stddef.h> // size_t
   #include <stdio.h> // printf
   #include "crc_in_rust.h"
   
   int main() { 
       uint8_t data[] = { 0,1,2,3,4,5,6 };
       size_t data_length = 7;
   
       uint32_t hash = crc32(data, data_length);
   
       printf("Hash: %d\n", hash);
   
       return 0;
   }
   

5. compile and run

   $ clang main.c target/debug/libcrc_in_rust.so -omain
   $ ./main
   Hash: -1386739207
   

C3.3 Bindgen

Use bindgen to generate the FFI bindings. Bindgen will look at a C header file, and generate rust functions, types and constants based on the C definitions.

But the generated code is ugly and non-idiomatic. To wrap a C library properly, good API design and documentation is needed.

We'll be making rust bindings for the tweetnacl C library. Goal: implement crypto_hash_sha256_tweet

Below you find instructions for using bindgen and wrapping crypto_hash_sha512_tweet. Follow the instructions, then repeat the steps for crypto_hash_sha256_tweet

Steps

1. Have a look at build.rs and src/lib.rs to get an idea of how to configure bindgen. Can you explain in your own words how this setup works?

You can refer to The bindgen User Guide for information on how to use bindgen

2. Run cargo check to verify everything is compiling correctly.

Inspecting our bindings

The output of cargo check contains a line with the bindings.rs path. Open that file.

In the generated bindings.rs file we find this signature for the crypto_hash_sha512_tweet C function from tweetNaCl:

#![allow(unused)]
fn main() {
extern "C" {
    pub fn crypto_hash_sha512_tweet(
        arg1: *mut ::std::os::raw::c_uchar,
        arg2: *const ::std::os::raw::c_uchar,
        arg3: ::std::os::raw::c_ulonglong,
    ) -> ::std::os::raw::c_int;
}
}

Some observations

• The definition is inside of an extern "C" block, and has no body. Therefore this function is marked as an extern, and rust expects it to be linked in.
• The function is marked pub, meaning we can import and use it in other modules (like main.rs in our case)
• We can deduce the behavior from the type signature:
  • arg1 is the output: a mutable pointer to a sequence of bytes
  • arg2 is the input: a constant pointer to a sequence of bytes
  • arg3 is a length (unclear of what)
  • the return value is probably an error code
• These are raw C types, which makes it a hassle to call directly from rust.

We will deal with the last point by writing some nice rust wrappers around the generated bindings.

In rust we bundle a pointer to a sequence of elements and its length in a slice. We could write the signature of our own rust wrapper function as:

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(out: &mut [u8], data: &[u8]) -> i32 {
    todo!()
}
}

Modelling with types

But by looking at the tweetNaCl source code we can see that the contract is a bit stronger:

• the output is always 64 bytes wide (64 * 8 = 512)
• we only ever return 0

int crypto_hash(u8 *out,const u8 *m,u64 n)
{
  u8 h[64],x[256];
  u64 i,b = n;

  FOR(i,64) h[i] = iv[i];

  crypto_hashblocks(h,m,n);
  m += n;
  n &= 127;
  m -= n;

  FOR(i,256) x[i] = 0;
  FOR(i,n) x[i] = m[i];
  x[n] = 128;

  n = 256-128*(n<112);
  x[n-9] = b >> 61;
  ts64(x+n-8,b<<3);
  crypto_hashblocks(h,x,n);

  FOR(i,64) out[i] = h[i];

  return 0;
}

The rust type system can model these invariants: We can explicitly make the output 64 elements long by using a reference to an array. Furthermore we can drop the return type if there is nothing useful to return.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(out: &mut [u8; 64], data: &[u8]) {
    todo!()
}
}

But even better, we can return the output array directly:

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    todo!()
}
}

The compiler will turn this signature into the one we had before under the hood. Returning the value is more idiomatic and convenient in rust, and with modern compilers there is no performance penalty.

In detail: The C ABI mandates that any return value larger than those that fit in a register (typically 128 bits nowadays) are allocated on the caller's stack. The first argument to the function is the pointer to write the result into. LLVM, the backend used by the rust compiler has specific optimizations to make sure the function result is written directly into this pointer.

Writing our implementation

Allright, with the signature worked out, we can write the actual implementation.

We can reach the bindings from main.rs with e.g.

#![allow(unused)]
fn main() {
tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(a,b,c);
}

Here tweetnacl_bindgen is the name of the project, specified in the package section of the Cargo.toml

[package]
name = "tweetnacl-bindgen"

Then bindings is the module name (the file src/bindings.rs is implicitly also a module) and finally crypto_hash_sha512_tweet is the function name from the original C library.

On to the implmentation. Extern functions are considered unsafe in rust, so we will need an unsafe block to call ours.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            todo!(),
            todo!(),
            todo!(),
        );
    }
}
}

Next we can pass our argument: we turn the slice into a pointer with .as_ptr(), and get the length with len(). The length needs to be cast to the right type. In this case we can use as _ where rust will infer the right type to cast to.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            todo!(),
            data.as_ptr(),
            data.len() as _,
        );
    }
}
}

Next we create an array for the return value, pass a mutable pointer to this memory to our extern functin, and return the array.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    let mut result = [ 0; 64 ];

    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            &mut result as *mut _,
            data.as_ptr(),
            data.len() as _,
        );
    }

    result
}
}

And we're done: an idiomatic rust wrapper around the crypto_hash_sha512_tweet!

Uninitialized memory

There is one more trick: our current function initializes and zeroes out the memory for result. That is wasteful because the extern function will overwrite these zeroes. Because the extern function is linked in, the compiler likely does not have enough information to optimize the zeroing out away.

The solution is MaybeUninit:

#![allow(unused)]
fn main() {
use std::mem::MaybeUninit;

pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    let mut result : MaybeUninit<[u8; 64]> = MaybeUninit::uninit();

    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            result.as_mut_ptr() as *mut _,
            data.as_ptr(),
            data.len() as _,
        );

        result.assume_init()
    }
}
}

The std::mem::MaybeUninit type is an abstraction for uninitialized memory. The .uninit() method gives a chunk of uninitialized memory big enough to store a value of the desired type (in our case [u8; 64] will be inferred).

We can look at the LLVM IR to verify that 1) the initialization with zeroes is not optimized away and 2) using MaybeUninit does not initialize the array.

Below is a call site of our crypto_hash_sha512_tweet function that zeroes out the memory. Indeed, we see a memset that sets all the bytes to 0. (also not that our wrapper function actually got inlined)

%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(64) %0, i8 0, i64 64, i1 false), !alias.scope !8, !noalias !11
%_2.i = call i32 @bindings::crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull "foobarbaz", i64 9)

In constrast, the version with MaybeUninit just calls our extern function without touching the memory at all:

%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0

%_3.i = call i32 @bindings::crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull "foobarbaz", i64 9), !noalias !6

define i8 @call_with_maybeuninit() unnamed_addr #1 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
start:
  %result.i = alloca <64 x i8>, align 1
  %0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 64, i8* nonnull %0), !noalias !2
  %_3.i = call i32 @crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull getelementptr inbounds (<{ [9 x i8] }>, <{ [9 x i8] }>* @alloc1, i64 0, i32 0, i64 0), i64 9), !noalias !6
  %1 = load <64 x i8>, <64 x i8>* %result.i, align 1, !noalias !7
  call void @llvm.lifetime.end.p0i8(i64 64, i8* nonnull %0), !noalias !2
  %2 = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> %1)
  ret i8 %2
}

define i8 @call_without_maybeuninit() unnamed_addr #1 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
start:
  %_4 = alloca <64 x i8>, align 1
  %0 = getelementptr inbounds <64 x i8>, <64 x i8>* %_4, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 64, i8* nonnull %0)
  call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(64) %0, i8 0, i64 64, i1 false), !alias.scope !8, !noalias !11
  %_2.i = call i32 @crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull getelementptr inbounds (<{ [9 x i8] }>, <{ [9 x i8] }>* @alloc1, i64 0, i32 0, i64 0), i64 9)
  %1 = load <64 x i8>, <64 x i8>* %_4, align 1
  %2 = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> %1)
  call void @llvm.lifetime.end.p0i8(i64 64, i8* nonnull %0)
  ret i8 %2
}

Wrap-up

Slides

Evaluation form

Thank you for taking the time to help us improve the workshop!

You can find the evaluation from here