msgspec-patterns

Use Structs for Structured Data

Always prefer msgspec.Struct over dict, dataclasses, or attrs for structured data with a known schema. Structs are 5-60x faster for common operations and are optimized for encoding/decoding.

# BAD
from dataclasses import dataclass

@dataclass
class User:
    name: str
    email: str
    age: int

# GOOD
import msgspec

class User(msgspec.Struct):
    name: str
    email: str
    age: int

user = User(name="alice", email="[email protected]", age=30)
data = msgspec.json.encode(user)
decoded = msgspec.json.decode(data, type=User)

Struct Configuration Options

Omit Default Values

Set omit_defaults=True when default values are known on both encoding and decoding ends. Reduces encoded message size and improves performance.

class Config(msgspec.Struct, omit_defaults=True):
    host: str = "localhost"
    port: int = 8080
    debug: bool = False

config = Config(host="production.example.com")
msgspec.json.encode(config)
# b'{"host":"production.example.com"}' — port and debug omitted

Avoid Decoding Unused Fields

Define smaller "view" Struct types that only contain the fields you actually need. msgspec skips decoding fields not defined in your Struct, reducing allocations and CPU time.

# BAD: decodes entire object
class FullTweet(msgspec.Struct):
    id: int
    full_text: str
    user: dict
    entities: dict
    retweet_count: int
    favorite_count: int
    # ... many more fields

# GOOD: only these fields are decoded, the rest is skipped
class User(msgspec.Struct):
    name: str

class TweetView(msgspec.Struct):
    user: User
    full_text: str
    favorite_count: int

array_like=True

Set array_like=True when both ends know the field schema. Encodes structs as arrays instead of objects, removing field names from the message — smaller and faster.

class Point(msgspec.Struct, array_like=True):
    x: float
    y: float
    z: float

point = Point(1.0, 2.0, 3.0)
msgspec.json.encode(point)
# b'[1.0,2.0,3.0]' instead of b'{"x":1.0,"y":2.0,"z":3.0}'

Tagged Unions

Use tag=True on Struct types when handling multiple message types in a single union for efficient type discrimination during decoding.

class GetRequest(msgspec.Struct, tag=True):
    key: str

class PutRequest(msgspec.Struct, tag=True):
    key: str
    value: str

class DeleteRequest(msgspec.Struct, tag=True):
    key: str

Request = GetRequest | PutRequest | DeleteRequest
decoder = msgspec.msgpack.Decoder(Request)

data = msgspec.msgpack.encode(PutRequest(key="foo", value="bar"))
request = decoder.decode(data)

match request:
    case GetRequest(key):    print(f"Get: {key}")
    case PutRequest(key, value): print(f"Put: {key}={value}")
    case DeleteRequest(key): print(f"Delete: {key}")

Use encode_into for Buffer Reuse

In hot loops, use Encoder.encode_into() with a pre-allocated bytearray instead of encode() to avoid allocating a new bytes object per call. Always measure before adopting.

# BAD: new bytes object allocated each iteration
encoder = msgspec.msgpack.Encoder()
for msg in msgs:
    data = encoder.encode(msg)
    socket.sendall(data)

# GOOD: reuse a buffer
encoder = msgspec.msgpack.Encoder()
buffer = bytearray(1024)
for msg in msgs:
    n = encoder.encode_into(msg, buffer)
    socket.sendall(memoryview(buffer)[:n])

NDJSON with encode_into

For line-delimited JSON, use encode_into() to avoid the copy from string concatenation:

encoder = msgspec.json.Encoder()
buffer = bytearray(64)
for msg in messages:
    n = encoder.encode_into(msg, buffer)
    file.write(memoryview(buffer)[:n])
    file.write(b"\n")

Length-Prefix Framing

Use encode_into() with an offset to efficiently prepend a message length without extra copies:

def send_length_prefixed(socket, msg):
    encoder = msgspec.msgpack.Encoder()
    buffer = bytearray(64)
    n = encoder.encode_into(msg, buffer, 4)   # leave 4 bytes at front
    buffer[:4] = n.to_bytes(4, "big")
    socket.sendall(memoryview(buffer)[:4 + n])

async def prefixed_send(stream, buffer: bytes) -> None:
    stream.write(len(buffer).to_bytes(4, "big"))
    stream.write(buffer)
    await stream.drain()

async def prefixed_recv(stream) -> bytes:
    prefix = await stream.readexactly(4)
    n = int.from_bytes(prefix, "big")
    return await stream.readexactly(n)

Use MessagePack for Internal APIs

msgspec.msgpack is more compact and can be more performant than msgspec.json for internal service communication.

class Event(msgspec.Struct):
    type: str
    data: dict
    timestamp: float

encoder = msgspec.msgpack.Encoder()
decoder = msgspec.msgpack.Decoder(Event)
packed = encoder.encode(Event(type="login", data={"user_id": 123}, timestamp=1703424000.0))

TOML Configuration Files

Use msgspec for parsing pyproject.toml and other TOML config files with validation:

class BuildSystem(msgspec.Struct, omit_defaults=True, rename="kebab"):
    requires: list[str] = []
    build_backend: str | None = None

class Project(msgspec.Struct, omit_defaults=True, rename="kebab"):
    name: str | None = None
    version: str | None = None
    dependencies: list[str] = []

class PyProject(msgspec.Struct, omit_defaults=True, rename="kebab"):
    build_system: BuildSystem | None = None
    project: Project | None = None
    tool: dict[str, dict[str, Any]] = {}

def load_pyproject(path: str) -> PyProject:
    with open(path, "rb") as f:
        return msgspec.toml.decode(f.read(), type=PyProject)

gc=False — Safety Analysis

Setting gc=False on a Struct means instances are never tracked by Python's garbage collector. This reduces GC pressure (up to 75x less GC pause time, 16 bytes saved per instance). The only risk: if a reference cycle involves only gc=False structs, that cycle will never be collected — memory leak.

Reference: msgspec Structs – Disabling Garbage Collection

When to use this analysis

• Adding or modifying a class that inherits from msgspec.Struct
• Reviewing or refactoring code that defines or uses msgspec structs
• Deciding whether to add or remove gc=False on a Struct

Verified safety constraints

All of the following must hold to use gc=False safely.

1. No reference cycles

• The struct (and any container it references) must never be part of a reference cycle.
• Multiple variables pointing to the same struct (x = s; y = x) are safe — that is not a cycle. A cycle is A → B → … → A.
• Returning a struct from a function is safe. What matters is whether any reference path leads back to the struct.

2. No mutation that could create cycles

• Do not mutate struct fields after construction in a way that could introduce a cycle (e.g. set a field to an object that references the struct, or append the struct to its own list/dict).
• Frozen structs (frozen=True) prevent field reassignment; force_setattr in __post_init__ is one-time init only — acceptable.
• Assigning scalars (int, str, bool, float, None) to fields is always safe.

3. Mutable containers (list, dict, set) on the struct

• If the struct has list/dict/set fields, either:
  • Never mutate those containers after creation and never store in them any object that references the struct, or
  • Do not use gc=False (conservative).
• Reading from containers does not create cycles and is always allowed.

4. Nested structs

• If a struct holds another Struct (or containers that hold Structs), the same rules apply to the whole reference graph. No cycles, no mutation that could create cycles.

5. Generic / mixins

• With gc=False, the type must be compatible with __slots__ (e.g. if using Generic, the mixin must define __slots__ = ()). See msgspec issue #631 / PR #635.

Decision tree

Should I use gc=False?
│
├── Does your Struct only contain scalar types (int, float, str, bool, bytes)?
│   └── YES → SAFE
│
├── Does your Struct contain lists/dicts and you control what goes in them?
│   └── Will you EVER put the struct itself (or a parent) into those containers?
│       ├── NO → Probably safe, but audit carefully
│       └── YES/MAYBE → Do NOT use gc=False
│
├── Does your Struct reference another Struct of the same type (tree, linked list)?
│   └── YES → Do NOT use gc=False
│
├── Is your Struct part of a bidirectional parent-child relationship?
│   └── YES → Do NOT use gc=False
│
└── When in doubt → Do NOT use gc=False

E

Content truncated.