MsgSpec¶

Prior to version 2.4.0, as well known, Esmerald was using Pydantic to make the life easier for almost everyone using the framework but things evolved and it was necessary to support other ways to validate data too, msgspec became predominant and widely adopted by the community and therefore it made sense also to provide support in Esmerald.

from esmerald.datastructures.msgspec import Struct

Warning

For a full integration of Esmerald with OpenAPI this is mandatory to be used. Using msgspec.Struct for OpenAPI models will incur in errors.

esmerald.datastructures.msgspec.Struct is exactly the same as msgspec.Struct but with extras added for OpenAPI purposes.

What is `msgspec`¶

As their documentation mention:

msgspec is a fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML.

msgspec and Pydantic are two extremely powerful libraries and both serve also different purposes but there are a lot of people that prefer msgspec to Pydantic for its performance.

A good example, as per msgspec documentation.

import msgspec

class User(msgspec.Struct):
    """A new type describing a User"""
    name: str
    groups: set[str] = set()
    email: str | None = None

`msgspec` and Esmerald¶

Esmerald supports msgspec with the nuances of what the framework can offer without breaking any native functionality.

So what does this mean? Well, Esmerald for OpenAPI documentation uses internal processes that rely on Pydantic and that should remain (at least for now) but also integrates msgspec in a seemless way.

This means, when you implement msgspec structs in your code, the error handling is delegated to the library and no Pydantic is involved, for obvious reasons, as well as the serialisation and deserialization of the data.

The nuances of Esmerald¶

This is what Esmerald can also do for you. By nuances, what Esmerald actually means is that you can mix msgspec structs within your Pydantic models but not the other way around.

Is this useful? Depends of what you want to do. You probably won't be using this but it is there in case you feel like playing around.

Nothing to worry about, we will be covering this in detail.

Importing `msgspec`¶

As mentioned at the very top of this document, to import the msgspec module you will need to:

from esmerald.datastructures.msgspec import Struct

Now, this can be a bit confusing, right? Why we need to import from this place instead of using directly the msgspec.Struct?

Well, actually you can use directly the msgspec.Struct but as mentioned before, Esmerald uses Pydantic for the OpenAPI documentation and to add the nuances we all love and therefore the esmerald.datastructures.msgspec.Struct is simply an extended object that adds some Pydantic flavours for the OpenAPI.

This also means that you can declare OpenAPIResponses using msgspec. Pretty cool, right?

Warning

If you don't use the esmerald.datastructures.msgspec.Struct, it won't be possible to use the msgspec with Pydantic. At least not in a cleaner supported way.

How to use it¶

`esmerald.datastructures.msgspec.Struct`¶

Well, let us see how we would work with msgspec inside Esmerald.

In a nutshell, it is exactly the same as you would normally do if you were creating a Pydantic base model or a datastructure to be used within your application.

from typing import Union

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct


class User(Struct):
    name: str
    email: Union[str, None] = None


@post()
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

Simple, right? Yes and there is a lot going here.

You you might have noticed, we are importing from esmerald.datastructures.msgspec the Struct and this is a good habit to have if you care about OpenAPI documentation and then we are simply declaring the data as the User of type Struct as the data in and as a response the User as well.

The reason why we declare the User as a response it is just to show that msgspec can also be used as another Esmerald response.

The rest, it is still as clean as always was in Esmerald.

Now, the cool part is when we send a payload to the API, something like this:

data = {"name": "Esmerald", "email": "esmerald@esmerald.dev"}

When this payload is sent, the validations are done automatically by the msgspec library which means you can implement as many validations as you want as you would normally do while using msgspec.

from typing import Union

import msgspec
from typing_extensions import Annotated

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct

Name = Annotated[str, msgspec.Meta(min_length=5)]
Email = Annotated[str, msgspec.Meta(min_length=5, max_length=100, pattern="[^@]+@[^@]+\\.[^@]+")]


class User(Struct):
    name: Name
    email: Union[Email, None] = None


@post()
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

And just like that, you are now using msgspec and all of its power within Esmerald.

`msgspec.Struct`¶

As mentioned before, importing from esmerald.datastructures.msgspec.Struct should be the way for Esmerald to use it without any issues but you can still use the normal msgspec.Struct as well.

from typing import Union

import msgspec

from esmerald import Esmerald, Gateway, post


class User(msgspec.Struct):
    name: str
    email: Union[str, None] = None


@post()
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

Now this is possible and it will work as normal but it comes with limitations. When accessing the OpenAPI, it will raise errors.

Again, esmerald.datatructures.msgspec.Struct is exactly the same as the msgspec.Struct with extra Esmerald flavours and this means you will not have the problem of updating the version of msgspec anytime you need.

Nested structs¶

Well, this is now what you can do already with msgspec and not directly related with Esmerald but for example purposes, let us see how it would look like it having a nested Struct.

from typing import Union

import msgspec
from typing_extensions import Annotated

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct

Name = Annotated[str, msgspec.Meta(min_length=5)]
Email = Annotated[str, msgspec.Meta(min_length=5, max_length=100, pattern="[^@]+@[^@]+\\.[^@]+")]
PostCode = Annotated[str, msgspec.Meta(min_length=5)]


class Address(Struct):
    post_code: PostCode
    street_address: Union[str, None] = None


class User(Struct):
    name: Name
    email: Union[Email, None] = None
    address: Address


@post()
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

One possible payload would be:

{
    "name": "Esmerald",
    "email": "esmerald@esmerald.dev",
    "address": {
        "post_code": "90210",
        "street_address": "California"
    }
}

The nuances of `Struct`¶

It was mentioned numerous times the use of esmerald.datastructures.msgspec.Struct and what it could give you besides the obvious needed OpenAPI documentation.

This is not the only thing. This special datastructure also allows you to mix with Pydantic models if you want to.

Does that mean the Struct will be then evaluated by Pydantic? No, it does not.

The beauty of this system is that every Struct/BaseModel is evaluated by its own library which means that if you have a Struct inside a BaseModel, the validations are done separately.

Why would you mix them if they are different? Well, in theory you wouldn't but you will never know what people want so Esmerald offers that possibility but not the other way around.

Let us see how it would look like having both working side by side.

from typing import Union

import msgspec
from pydantic import BaseModel, EmailStr
from typing_extensions import Annotated

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct

StreetAddress = Annotated[str, msgspec.Meta(min_length=5)]
PostCode = Annotated[str, msgspec.Meta(min_length=5)]


class Address(Struct):
    post_code: PostCode
    street_address: Union[StreetAddress, None] = None


class User(BaseModel):
    name: str
    email: Union[EmailStr, None] = None
    address: Address


@post()
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

This works perfectly well and the payload is still like this:

{
    "name": "Esmerald",
    "email": "esmerald@esmerald.dev",
    "address": {
        "post_code": "90210",
        "street_address": "California"
    }
}

The difference is that the address part of the payload will be evaluated by msgspec and the rest by Pydantic.

Responses¶

As mentioned before, the Struct of msgspec can also be used as Response of Esmerald. This will enable the internal mechanisms to serialize/deserialize with the power of the native library as everyone came to love.

You can see more details about the types of responses you can use with Esmerald.

OpenAPI Documentation¶

Now this is the nice part of msgspec. How can you integrate msgspec with OpenAPI?

Well, as mentioned before, in the same way you would normally do. You can also use OpenAPIResponse with the Struct as well!

Let us see the previous example again.

from typing import Union

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct


class User(Struct):
    name: str
    email: Union[str, None] = None


@post()
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

This will generate a simple OpenAPI documentation using the Struct.

What if you want to create OpenAPIResponse objects? Well, there are also three ways:

As single object.
As a list.
Mixing with Pydantic (the nuance).

As a single object¶

from typing import Union

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct
from esmerald.openapi.datastructures import OpenAPIResponse


class ErrorDetail(Struct):
    detail: str
    code: int


class BadRequest(Struct):
    detail: str
    field: str
    code: int


class User(Struct):
    name: str
    email: Union[str, None] = None


@post(
    summary="Creates a user in the system",
    responses={
        400: OpenAPIResponse(model=ErrorDetail),
        401: OpenAPIResponse(model=BadRequest),
    },
)
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

As a list¶

from typing import Union

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct
from esmerald.openapi.datastructures import OpenAPIResponse


class ErrorDetail(Struct):
    detail: str
    code: int


class User(Struct):
    name: str
    email: Union[str, None] = None


@post(
    summary="Creates a user in the system",
    responses={
        400: OpenAPIResponse(model=[ErrorDetail]),
    },
)
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

Mixing with Pydantic¶

from typing import List, Union

from pydantic import BaseModel

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct
from esmerald.openapi.datastructures import OpenAPIResponse


class ErrorDetail(Struct):
    detail: str
    code: int


class Error(BaseModel):
    errors: List[ErrorDetail]


class User(Struct):
    name: str
    email: Union[str, None] = None


@post(
    summary="Creates a user in the system",
    responses={
        400: OpenAPIResponse(model=Error),
    },
)
def create(data: User) -> User:
    """
    Returns the same payload sent to the API.
    """
    return data


app = Esmerald(routes=[Gateway(handler=create)])

Notes¶

This is the integration with msgspec and Esmerald in a simple fashion where you can take advantage of the powerful msgspec library and the elegance of Esmerald.

This section covers the dos and dont's of the Struct and once again, use:

from esmerald.datastructures.msgspec import Struct

And have fun!

MsgSpec¶

What is msgspec¶

msgspec and Esmerald¶