Golang Structs Memory Allocation

All values and examples have been run on Go Playground using Go 1.21

In Golang the memory allocation follows a set of rules. Before getting into those rules we need to understand what the alignment of variables is

Golang unsafe package has a function Alignof, with signature

func Alignof(x ArbitraryType) uintptr

for any variable x of type v passed to AlignOf returns the alignment for that variable. Let's call the alignment m. Now, Go ensures that m is the largest possible number for which memory address of variable x % m == 0 holds true, i.e. memory address of variable x is a multiple of m.

Let's look at the alignment values for some data types.

byte, int8, uint8 -> 1
int16, uint16 -> 2
int32, uint32, float32, complex64 -> 4
int, int64, uint64, float64, complex128 -> 8

string, slice -> 8

The behavior might be different for a field within the struct, the details for which are in the package documentation.

To better understand memory allocation happening within a struct, we will make use of one more function of the unsafe package, and that is Offsetof. This function returns the location of the starting position of a field relative to the starting position of the struct, in other words, the number of bytes between the start of the field and the start of the struct.

func Offsetof(x ArbitraryType) uintptr

To better understand the memory allocation inside a struct, let's take an example of a struct below

type Example struct {
    a int8
    b string
    c int8
    d int32
}

now we will find out the total memory taken by a variable of type Example and try to optimize the allocation.

    var v = Example{
        a: 10,
        b: "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus rhoncus."
        c: 20,
        d: 100,
    }
    fmt.Println("Offset of a", unsafe.Offsetof(v.a)) // 0
    fmt.Println("Offset of b", unsafe.Offsetof(v.b)) // 8
    fmt.Println("Offset of c", unsafe.Offsetof(v.c)) // 24
    fmt.Println("Offset of d", unsafe.Offsetof(v.d)) // 28

Now, the question arises, "Why is the offset of field b 8 for this struct? It should have been 1 because the field a is of type int8 which is only 1 byte in size. Going back to the alignment of string data types, the value for which is 8, means that the address needs to be divisible by 8 and hence a "padding" of 7 bytes is added in between to ensure this behavior.

Why is the offset of field c 24? The string in field b looks a lot longer than 16 bytes and if the offset of string is 8, field c should have a slightly larger offset.

The answer to the above question is that in Go the string is not allocated memory in the same place within the struct. There is a separate data structure that holds the string descriptor and this string descriptor is stored in-place in the struct for the field of type string, and the size of that descriptor is 16 bytes.

Now, let's look at another function in unsafe package called Sizeof. This function as the name suggests estimates and returns the number of bytes a variable of type x will hold.

Note: it estimates the size because in a struct there can be fields of varying memory size.

func Sizeof(x ArbitraryType) uintptr

Now, let's look at the Size of our struct Example

    fmt.Println("Sizeof Example", unsafe.Sizeof(v)) // 32

How can we optimize this struct to minimize padding?

To get into memory optimization for this struct, we will look at the alignment values of different data types and try to minimize padding. Let's try keeping both fields of type int8 together

type y struct {
        a int8
        c int8
        b string
        d int32
    }

    var v = y{}
    fmt.Println("Offset of a", unsafe.Offsetof(v.a)) // 0
    fmt.Println("Offset of b", unsafe.Offsetof(v.b)) // 8
    fmt.Println("Offset of c", unsafe.Offsetof(v.c)) // 1
    fmt.Println("Offset of d", unsafe.Offsetof(v.d)) // 24
    fmt.Println("Sizeof Example", unsafe.Sizeof(v)) // 32

Yay, we got rid of some padding, but why is the size still 32? The size should have been 1(a) + 1(c) + 6(padding) + 16(b) + 4(d) = 28

Now, when the last field of a struct doesn't align perfectly with the architecture's alignment requirements, padding is added after the last field to ensure that the overall size of the struct is a multiple of the largest alignment requirement among its fields. Because the largest alignment is 8 for string data type, additional padding was added to make the size a multiple of 8, i.e. 4 more bytes padded at the end to make the size 32 bytes.

Can we further get rid of padding and make this even more optimized?

Let's give it a try by moving field locations.

type y struct {
        b string
        d int32
        a int8
        c int8
    }

    var v = y{}
    fmt.Println("Offset of a", unsafe.Offsetof(v.a)) // 20
    fmt.Println("Offset of b", unsafe.Offsetof(v.b)) // 0
    fmt.Println("Offset of c", unsafe.Offsetof(v.c)) // 21
    fmt.Println("Offset of d", unsafe.Offsetof(v.d)) // 16
    fmt.Println("Sizeof Example", unsafe.Sizeof(v)) // 24

We can see that by reordering the fields so that the alignment needs minimal padding, we have been able to reduce the size of the struct to 24 from 32, which is a massive 25% optimization in memory.

The current memory footprint is 16(b) + 4(d) + 1(a) + 1(b) + 2(padding).

Alas, we won't be able to get rid of more padding due to the constraints in the language and architecture.

Part II of the article here: link