In this article by Ivo Balbaert, author of the book, Rust Essentials, we will go through the pointers and memory safety.
(For more resources related to this topic, see here.)
The stack and the heap
When a program starts, by default a 2 MB chunk of memory called the stack is granted to it. The program will use its stack to store all its local variables and function parameters; for example, an i32 variable takes 4 bytes of the stack. When our program calls a function, a new stack frame is allocated to it. Through this mechanism, the stack knows the order in which the functions are called so that the functions return correctly to the calling code and possibly return values as well.
Dynamically sized types, such as strings or arrays, can’t be stored on the stack. For these values, a program can request memory space on its heap, so this is a potentially much bigger piece of memory than the stack.
When possible, stack allocation is preferred over heap allocation because accessing the stack is a lot more efficient.
Lifetimes
All variables in a Rust code have a lifetime. Suppose we declare an n variable with the let n = 42u32; binding. Such a value is valid from where it is declared to when it is no longer referenced, which is called the lifetime of the variable. This is illustrated in the following code snippet:
fn main() {
let n = 42u32;
let n2 = n; // a copy of the value from n to n2
life(n);
println!("{}", m); // error: unresolved name `m`.
println!("{}", o); // error: unresolved name `o`.
}
fn life(m: u32) -> u32 {
let o = m;
o
}
The lifetime of n ends when main() ends; in general, the start and end of a lifetime happen in the same scope. The words lifetime and scope are synonymous, but we generally use the word lifetime to refer to the extent of a reference. As in other languages, local variables or parameters declared in a function do not exist anymore after the function has finished executing; in Rust, we say that their lifetime has ended. This is the case for the m and o variables in the preceding code snippet, which are only known in the life function.
Likewise, the lifetime of a variable declared in a nested block is restricted to that block, like phi in the following example:
{
let phi = 1.618;
}
println!("The value of phi is {}", phi); // is error
Trying to use phi when its lifetime is over results in an error: unresolved name ‘phi’.
The lifetime of a value can be indicated in the code by an annotation, for example ‘a, which reads as lifetime where a is simply an indicator; it could also be written as ‘b, ‘n, or ‘life. It’s common to see single letters being used to represent lifetimes. In the preceding example, an explicit lifetime indication was not necessary since there were no references involved. All values tagged with the same lifetime have the same maximum lifetime.
In the following example, we have a transform function that explicitly declares the lifetime of its s parameter to be ‘a:
fn transform<'a>(s: &'a str) { /* ... */ }
Note the <‘a> indication after the name of the function. In nearly all cases, this explicit indication is not needed because the compiler is smart enough to deduce the lifetimes, so we can simply write this:
fn transform_without_lifetime(s: &str) { /* ... */ }
Here is an example where even when we indicate a lifetime specifier ‘a, the compiler does not allow our code. Let’s suppose that we define a Magician struct as follows:
struct Magician {
name: &'static str,
power: u32
}
We will get an error message if we try to construct the following function:
fn return_magician<'a>() -> &'a Magician {
let mag = Magician { name: "Gandalf", power: 4625};
&mag
}
The error message is error: ‘mag’ does not live long enough. Why does this happen? The lifetime of the mag value ends when the return_magician function ends, but this function nevertheless tries to return a reference to the Magician value, which no longer exists. Such an invalid reference is known as a dangling pointer. This is a situation that would clearly lead to errors and cannot be allowed.
The lifespan of a pointer must always be shorter than or equal to than that of the value which it points to, thus avoiding dangling (or null) references.
In some situations, the decision to determine whether the lifetime of an object has ended is complicated, but in almost all cases, the borrow checker does this for us automatically by inserting lifetime annotations in the intermediate code; so, we don’t have to do it. This is known as lifetime elision.
For example, when working with structs, we can safely assume that the struct instance and its fields have the same lifetime. Only when the borrow checker is not completely sure, we need to indicate the lifetime explicitly; however, this happens only on rare occasions, mostly when references are returned.
One example is when we have a struct with fields that are references. The following code snippet explains this:
struct MagicNumbers {
magn1: &u32,
magn2: &u32
}
This won’t compile and will give us the following error: missing lifetime specifier [E0106].
Therefore, we have to change the code as follows:
struct MagicNumbers<'a> {
magn1: &'a u32,
magn2: &'a u32
}
This specifies that both the struct and the fields have the lifetime as ‘a.
Perform the following exercise:
Explain why the following code won’t compile:
fn main() {
let m: &u32 = {
let n = &5u32;
&*n
};
let o = *m;
}
Answer the same question for this code snippet as well:
let mut x = &3;
{
let mut y = 4;
x = &y;
}
Copying values and the Copy trait
In the code that we discussed in earlier section the value of n is copied to a new location each time n is assigned via a new let binding or passed as a function argument:
let n = 42u32;
// no move, only a copy of the value:
let n2 = n;
life(n);
fn life(m: u32) -> u32 {
let o = m;
o
}
At a certain moment in the program’s execution, we would have four memory locations that contain the copied value 42, which we can visualize as follows:
Each value disappears (and its memory location is freed) when the lifetime of its corresponding variable ends, which happens at the end of the function or code block in which it is defined. Nothing much can go wrong with this Copy behavior, in which the value (its bits) is simply copied to another location on the stack. Many built-in types, such as u32 and i64, work similar to this, and this copy-value behavior is defined in Rust as the Copy trait, which u32 and i64 implement.
You can also implement the Copy trait for your own type, provided all of its fields or items implement Copy. For example, the MagicNumber struct, which contains a field of the u64 type, can have the same behavior. There are two ways to indicate this:
- One way is to explicitly name the Copy implementation as follows:
struct MagicNumber { value: u64 } impl Copy for MagicNumber {} - Otherwise, we can annotate it with a Copy attribute:
#[derive(Copy)] struct MagicNumber { value: u64 }
This now means that we can create two different copies, mag and mag2, of a MagicNumber by assignment:
let mag = MagicNumber {value: 42};
let mag2 = mag;
They are copies because they have different memory addresses (the values shown will differ at each execution):
println!("{:?}", &mag as *const MagicNumber); // address is 0x23fa88
println!("{:?}", &mag2 as *const MagicNumber); // address is 0x23fa80
The *const function is a so-called raw pointer. A type that does not implement the Copy trait is called non-copyable.
Another way to accomplish this is by letting MagicNumber implement the Clone trait:
#[derive(Clone)]
struct MagicNumber {
value: u64
}
Then, we can use clone() mag into a different object called mag3, effectively making a copy as follows:
let mag3 = mag.clone();
println!("{:?}", &mag3 as *const MagicNumber); // address is 0x23fa78
mag3 is a new pointer referencing a new copy of the value of mag.
Pointers
The n variable in the let n = 42i32; binding is stored on the stack. Values on the stack or the heap can be accessed by pointers. A pointer is a variable that contains the memory address of some value. To access the value it points to, dereference the pointer with *. This happens automatically in simple cases such as in println! or when a pointer is given as a parameter to a method. For example, in the following code, m is a pointer containing the address of n:
let m = &n;
println!("The address of n is {:p}", m);
println!("The value of n is {}", *m);
println!("The value of n is {}", m);
This prints out the following output, which differs for each program run:
The address of n is 0x23fb34 The value of n is 42 The value of n is 42
So, why do we need pointers? When we work with dynamically allocated values, such as a String, that can change in size, the memory address of that value is not known at compile time. Due to this, the memory address needs to be calculated at runtime. So, to be able to keep track of it, we need a pointer for it whose value will change when the location of String in memory changes.
The compiler automatically takes care of the memory allocation of pointers and the freeing up of memory when their lifetime ends. You don’t have to do this yourself like in C/C++, where you could mess up by freeing memory at the wrong moment or at multiple times.
The incorrect use of pointers in languages such as C++ leads to all kinds of problems.
However, Rust enforces a strong set of rules at compile time called the borrow checker, so we are protected against them. We have already seen them in action, but from here onwards, we’ll explain the logic behind their rules.
Pointers can also be passed as arguments to functions, and they can be returned from functions, but the compiler severely restricts their usage.
When passing a pointer value to a function, it is always better to use the reference-dereference &* mechanism, as shown in this example:
let q = &42;
println!("{}", square(q)); // 1764
fn square(k: &i32) -> i32 {
*k * *k
}
References
In our previous example, m, which had the &n value, is the simplest form of pointer, and it is called a reference (or borrowed pointer); m is a reference to the stack-allocated n variable and has the &i32 type because it points to a value of the i32 type.
In general, when n is a value of the T type, then the &n reference is of the &T type.
Here, n is immutable, so m is also immutable; for example, if you try to change the value of n through m with *m = 7; you will get a cannot assign to immutable borrowed content ‘*m’ error. Contrary to C, Rust does not let you change an immutable variable via its pointer.
Since there is no danger of changing the value of n through a reference, multiple references to an immutable value are allowed; they can only be used to read the value, for example:
let o = &n;
println!("The address of n is {:p}", o);
println!("The value of n is {}", *o);
It prints out as described earlier:
The address of n is 0x23fb34 The value of n is 42
We could represent this situation in the memory as follows:
It is clear that working with pointers such as this or in much more complex situations necessitates much stricter rules than the Copy behavior. For example, the memory can only be freed when there are no variables or pointers associated with it anymore. And when the value is mutable, can it be changed through any of its pointers?
Mutable references do exist, and they are declared as let m = &mut n. However, n also has to be a mutable value. When n is immutable, the compiler rejects the m mutable reference binding with the error, cannot borrow immutable local variable ‘n’ as mutable. This makes sense since immutable variables cannot be changed even when you know their memory location.
To reiterate, in order to change a value through a reference, both the variable and its reference have to be mutable, as shown in the following code snippet:
let mut u = 3.14f64;
let v = &mut u;
*v = 3.15;
println!("The value of u is now {}", *v);
This will print: The value of u is now 3.15.
Now, the value at the memory location of u is changed to 3.15.
However, note that we now cannot change (or even print) that value anymore by using the u: u = u * 2.0; variable gives us a compiler error: cannot assign to ‘u’ because it is borrowed. We say that borrowing a variable (by making a reference to it) freezes that variable; the original u variable is frozen (and no longer usable) until the reference goes out of scope.
In addition, we can only have one mutable reference: let w = &mut u; which results in the error: cannot borrow ‘u’ as mutable more than once at a time. The compiler even adds the following note to the previous code line with: let v = &mut u; note: previous borrow of ‘u’ occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `u` until the borrow ends. This is logical; the compiler is (rightfully) concerned that a change to the value of u through one reference might change its memory location because u might change in size, so it will not fit anymore within its previous location and would have to be relocated to another address. This would render all other references to u as invalid, and even dangerous, because through them we might inadvertently change another variable that has taken up the previous location of u!
A mutable value can also be changed by passing its address as a mutable reference to a function, as shown in this example:
let mut m = 7;
add_three_to_magic(&mut m);
println!("{}", m); // prints out 10
With the function add_three_to_magic declared as follows:
fn add_three_to_magic(num: &mut i32) {
*num += 3; // value is changed in place through +=
}
To summarize, when n is a mutable value of the T type, then only one mutable reference to it (of the &mut T type) can exist at any time. Through this reference, the value can be changed.
Using ref in a match
If you want to get a reference to a matched variable inside a match function, use the ref keyword, as shown in the following example:
fn main() {
let n = 42;
match n {
ref r => println!("Got a reference to {}", r),
}
let mut m = 42;
match m {
ref mut mr => {
println!("Got a mutable reference to {}", mr);
*mr = 43;
},
}
println!("m has changed to {}!", m);
}
Which prints out:
Got a reference to 42 Got a mutable reference to 42 m has changed to 43!
The r variable inside the match has the &i32 type. In other words, the ref keyword creates a reference for use in the pattern. If you need a mutable reference, use ref mut.
We can also use ref to get a reference to a field of a struct or tuple in a destructuring via a let binding. For example, while reusing the Magician struct, we can extract the name of mag by using ref and then return it from the match:
let mag = Magician { name: "Gandalf", power: 4625};
let name = {
let Magician { name: ref ref_to_name, power: _ } = mag;
*ref_to_name
};
println!("The magician's name is {}", name);
Which prints: The magician’s name is Gandalf.
References are the most common pointer type and have the most possibilities; other pointer types should only be applied in very specific use cases.
Summary
In this article, we learned the intelligence behind the Rust compiler, which is embodied in the principles of ownership, moving values, and borrowing.
Resources for Article:
Further resources on this subject:
- Getting Started with NW.js [article]
- Creating Random Insults [article]
- Creating Man-made Materials in Blender 2.5 [article]
