Box to save memory in Rust

Table of Contents

I saved 475 MB out of the 895 MB used by a real-world Rust program by changing the layout of some structs and the way I was deserializing JSON files.

The real use case

My program deserializes all the JSON files of https://github.com/awslabs/aws-sdk-rust/tree/main/aws-models into "Smithy Shape" structs.

Those files contain thousands of structures similar to this one:

"com.amazonaws.iam#EnableOrganizationsRootSessionsResponse": {
"type": "structure",
"members": {
"OrganizationId": {
"target": "com.amazonaws.iam#OrganizationIdType",
"traits": {
"smithy.api#documentation": "<p>The unique identifier (ID) of an organization.</p>"
}
},
"EnabledFeatures": {
"target": "com.amazonaws.iam#FeaturesListType",
"traits": {
"smithy.api#documentation": "<p>The features you have enabled for centralized root access.</p>"
}
}
},
"traits": {
"smithy.api#output": {}
}
},

As is common in Rust, my program uses the very convenient serde.

I won't go into every details, but part of the structure needs to be shown at this point for clarity.

Don't read it entirely, just note that it's a bunch of structs containing structs, some optional, with serde attributes:

#[derive(Clone, Deserialize, Serialize)]
pub struct SmithyShape {
#[serde(rename = "type")]
pub shape_type: SmithyShapeType,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub operations: Vec<SmithyReference>,
#[serde(default)]
pub members: FxHashMap<String, SmithyReference>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub key: Option<SmithyReference>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub value: Option<SmithyReference>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub member: Option<SmithyReference>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub input: Option<SmithyReference>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub output: Option<SmithyReference>,
#[serde(default)]
pub traits: SmithyTraits,
}

This is some standard looking code, the current practice, but we can also call it naïve. By deserializing this way, the structures were taking 895MB in memory.

An analysis shows that most optional strings are missing, and that's what I leveraged to drastically reduce the memory footprint. But this requires to have in mind some Rust specificities, so a detour is needed:

About rust structs and memory

On a 64-bits platform, a word is made of 8 bytes. That's for example the memory needed to store a usize.

A String needs 3 words (address of the string, allocated size, and capacity) to which you need to add the allocated space for the string bytes. That's 24 bytes for a String, excluding the actual string content on the heap.

There's a niche compiler optimization which makes an Option<String> the same size (basically an option of a pointer type doesn't need an added byte to know whether it's None because it's None when the pointer is zero).

So the following structure, when all strings are missing (None), takes exactly 120 bytes (5*24) in memory:

pub struct SmithyServiceTrait {
pub sdk_id: Option<String>,
pub arn_namespace: Option<String>,
pub cloud_formation_name: Option<String>,
pub cloud_trail_event_source: Option<String>,
pub endpoint_prefix: Option<String>,
}

To allow our Rust Container to take only one word for the optional content when there's nothing to store, we need to put this content on the heap, outside of the container using Box.

The changes that recovered the memory

Basically, the change consists in:

• Detecting when structs are useless (i.e. when all their fields are None)
• Making them optional in their parent struct, and moving them to the heap using Box.
• Implementing a custom Deserializer to not store empty useless structs.

Similarly, SmithyShape was changed to replace all Option<SmithyReference> by Option<Box<SmithyReference>>, and that's how the memory needed to store all deserialized AWS shapes was reduced twofold, sparing 475 MB.

A few notes:

• this deserialization costs more in CPU.