Introducing ShapeShift: declare your data once, serialize it anywhere

Rust developers have been spoiled for years. You mark a type with #[derive(Serialize, Deserialize)], and the same struct can go to JSON, MessagePack, YAML, TOML, CBOR, and a long tail of other formats. The type describes what it is. Each format crate describes how to write bytes.

.NET has never had a first-class answer to that. We have excellent JSON serializers, excellent MessagePack serializers, YAML libraries, XML libraries — each with its own attributes, its own converters, its own idea of what a “required” property means. Change the wire format and you rewrite the contract.

ShapeShift is an experiment at closing that gap. One PolyType shape on your data. A shared converter layer that already knows about objects, collections, unions, naming policies, security limits, and schema. Format packages that are thin adapters over that layer. JSON, MessagePack, YAML, and TAML ship today. A third-party format is an encoder, a decoder, and a serializer that binds them together.

We are in alpha (targeting .NET 10). APIs will move. That is the point of saying so up front: please try it, break it, and tell us what hurt.

A type, then any format

Annotate the root of an object graph with PolyType’s [GenerateShape]. Nested types get shapes automatically.

using PolyType;
using ShapeShift.Json;
using ShapeShift.MsgPack;
using ShapeShift.Yaml;

[GenerateShape]
public partial record Person(string Name, int Age, List<string> Interests);

var person = new Person("Ada", 36, ["mathematics", "programming"]);

That is the entire contract. JSON:

var json = new JsonSerializer
{
    PropertyNamingPolicy = ShapeShiftNamingPolicy.CamelCase,
    Indented = true,
};

string text = json.Serialize(person);
Person? copy = json.Deserialize<Person>(text);

{
  "name": "Ada",
  "age": 36,
  "interests": [
    "mathematics",
    "programming"
  ]
}

The same value as MessagePack:

var msgpack = new MsgPackSerializer();
byte[] bytes = msgpack.Serialize(person);
Person? roundTripped = msgpack.Deserialize<Person>(bytes);

And as YAML:

var yaml = new YamlSerializer();
string document = yaml.Serialize(person);

Name: Ada
Age: 36
Interests:
  - mathematics
  - programming

No [JsonPropertyName]. No [Key(0)]. No parallel DTO hierarchy for the binary endpoint. Configuration — naming policy, default-value omission, required-member checks, depth and length limits, custom converters — lives on an immutable serializer instance and is shared across formats. JsonSerializer is a record; with derives a second configuration without mutating the first.

JsonSerializer compact = json with
{
    Indented = false,
    SerializeDefaultValues = SerializeDefaultValuesPolicy.Required,
};

Why PolyType is the interesting part

PolyType is Eirik Tsarpalis’s library for practical generic programming on .NET. A type shape is a compile-time description of a CLR type: its properties, constructors, collection construction, enum members, surrogates, unions. The source generator implements IShapeable<T> from [GenerateShape]. Anything that can walk a shape can consume your types — NativeAOT and trimming included, with no reflection in the default path.

That is a bigger idea than serialization.

In serde, a type implements Serialize / Deserialize. Those traits are serialization. A shape is not. The same Person shape can feed:

• JSON, MessagePack, YAML, TAML, or a format you wrote this afternoon
• a JSON Schema projection of the contract
• a structural equality comparer so two object graphs compare by value without hand-written Equals
• targeted deserialization: “skip to person.Address.City and decode only that”

JsonObject schema = json.GetJsonSchema<Person>();

IEqualityComparer<Person> byValue = StructuralEqualityComparer.Create<Person>();

Declare the data once. Use it for every operation that needs to understand the data, not only for writing JSON.

Reflection-based converter activation exists, but only behind an explicit opt-in (WithReflectionConverterTypes). If you never call it, the application stays NativeAOT-safe. There are no mutable process-wide serializer defaults.

serde, Serde.NET, and ShapeShift

The family resemblance is real. The architectures are not the same.

serde (Rust) split the world into data structures that know how to serialize themselves and data formats that know how to talk to those structures. #[derive(Serialize, Deserialize)] plus serde_json, rmp-serde, serde_yaml, and dozens of community format crates. It is the gold standard for “one type, many encodings,” and a large part of why Rust’s ecosystem feels coherent.

Serde.NET is a faithful port of that design to C#. [GenerateSerde] produces ISerialize / IDeserialize on your types. Formats implement ISerializer / IDeserializer. JSON ships in the box; MessagePack and XML are additional packages. It is source-generated, trim-safe, NativeAOT-friendly, and the closest thing .NET has had to “serde, but C#.” If you want serde’s programming model on .NET, that is the project to look at.

ShapeShift takes the goal and routes it through PolyType instead of through serialize traits:

In serde, each format still implements “write a struct,” “write a sequence,” “write an enum.” The type is shared; the object-mapping logic is not. ShapeShift pushes one layer further down. Objects, collections, unions, naming, default-value policy, required members, reference preservation, unknown-property retention, and contract inspection are implemented once, over an abstract token stream (StartMap, PropertyName, String, StartVector, …). A format package projects that stream onto bytes.

That is why ShapeShift.Taml can be small, and why the repo includes a complete UBJSON package written only against public APIs. You do not reimplement object mapping to invent a format. You implement an encoder and a decoder, run ShapeShift.Conformance, and the rest of the stack comes along.

Why this experiment exists: Nerdbank.MessagePack and Nerdbank.Json

This did not start as a green-field serializer. It started as déjà vu.

Nerdbank.MessagePack is a PolyType-based MessagePack library: NativeAOT-ready, source-generated contracts, analyzers, streaming and targeted deserialization, reference preservation, schema export, structural equality, strict security defaults. It is the specialized, performance-conscious MessagePack stack I actually recommend today.

Nerdbank.Json is the same idea for UTF-8 JSON: same PolyType shapes, same immutable serializer, same converter/factory/attribute story, same path-based targeting and unknown-member retention — implemented a second time against a JSON reader and writer.

Look at the two trees side by side and the duplication is obvious. Object-map converters and non-default-constructor converters. Collection and dictionary converters. Union and surrogate converters. Converter caches, factory pipelines, naming policies, default-value policy, security limits. Path expressions. Schema visitors. Structural equality. Analyzers that say “you forgot [GenerateShape].” The wire format changes. The interesting code does not.

Every new format on that trajectory would copy it again. Every bug fix would have to land twice. Every policy decision — what “required” means, whether duplicate properties are an error, how cycles are rejected — would drift.

ShapeShift is the experiment that asks: what if that layer were a library? Nerdbank.MessagePack and Nerdbank.Json stay the specialized, format-native serializers they are. ShapeShift is the shared core plus thin format packages, to see how much of that duplicated converter, policy, and diagnostics code can be written once and reused by JSON, MessagePack, YAML, TAML, and the next format someone actually needs.

If the experiment works, format authors get a much smaller job, and application authors get one set of attributes and one mental model. If it does not, we will have learned exactly where format-neutral abstractions leak. Either outcome is worth an alpha.

What you get in the alpha

Shared across every format:

• Records, primary constructors, init / required, mutable and immutable collections, dictionaries, nullable values, enums, surrogates, attributed unions
• Immutable serializer configuration; no mutable globals
• Custom converters, converter factories, and per-member converters
• Strict-by-default deserialization (duplicates, missing required members, non-nullable members) with configurable limits
• Optional default-value omission, reference preservation, string interning
• ShapeShiftValue for untyped trees, and extension-data for forward-compatible round trips
• Path-based targeted reads and streaming enumeration of sequences or concatenated documents
• Roslyn analyzers packed into the core package

Format packages on NuGet: ShapeShift.Json, ShapeShift.MsgPack, ShapeShift.Yaml, ShapeShift.Taml. Docs: getting started and the feature survey.

JSON is built on System.Text.Json primitives (Utf8JsonReader / Utf8JsonWriter). Object mapping is ShapeShift’s, not System.Text.Json.JsonSerializer’s. MessagePack implements the binary format directly, including timestamps, binary values, and an optional positional (array) contract when compactness matters more than version tolerance.

Please try it — and please complain

This is alpha. The target framework is net10.0. Public APIs are allowed to break. Performance is not the pitch yet; Nerdbank.MessagePack remains the library to beat for MessagePack specifically. Host integrations (ASP.NET Core MVC, SignalR) are deliberately deferred until the core is something we would want hosts to take a dependency on.

What we do want is people using it:

1. dotnet add package ShapeShift.Json --prerelease (or ShapeShift.MsgPack, ShapeShift.Yaml, …)
2. Put [GenerateShape] on a real DTO you already serialize
3. Round-trip it, then round-trip the same type through a second format
4. If you maintain a format, skim Authoring a format package and see whether the encoder/decoder split fits

Open an issue for missing types, surprising defaults, APIs that fight you, or a format you wish existed. “This should work like serde / like Nerdbank.MessagePack / like System.Text.Json in this case” is especially useful. The whole premise is that the shared layer can absorb those cases so the next format does not have to.

One type shape. Any encoding. That is the bet. Help us find out whether it holds.