Skip to main content
This document describes the architectural patterns for field tracking, dirty detection, and identity management in the StashObject base class.

Overview

Four critical architectural patterns are implemented in the StashObject base class:
  1. UUID4 Auto-Generation: New objects receive a temporary UUID4 identifier that is replaced with the server-assigned ID after save operations
  2. UNSET Sentinel Pattern: Three-level field system to distinguish between “set to value”, “set to null”, and “never touched”
  3. Field Tracking (_received_fields): Tracks which fields were actually loaded from GraphQL responses
  4. Dirty Tracking (_snapshot): Snapshot-based change detection for minimal update payloads
These patterns work together to enable:
  • Partial GraphQL fragments: Load only needed fields, track what was loaded
  • Minimal mutations: Send only changed fields to server
  • Null vs unset distinction: Explicitly set null vs never touched
  • Identity management: Track new vs existing objects with UUID transition

UUID4 Auto-Generation for New Objects

The Problem (Before)

Previously, new objects used the magic string "new" as a marker:
This had several issues:
  • Not type-safe (any string could be an ID)
  • Required explicit id="new" in every new object creation
  • Unclear intention (is “new” a valid ID or a marker?)

The Solution (After)

New objects automatically receive a UUID4 hex string (32 characters) when created without an ID:

UUID4 Methods

is_new() -> bool

Check if an object has a temporary UUID (not yet saved to server):
Detection Logic:
  • Returns True if ID is 32 hex characters (UUID4) and not all digits
  • Returns True if ID is the legacy "new" marker
  • Returns False for numeric IDs (typical server IDs)

update_id(server_id: str) -> None

Replace temporary UUID with server-assigned ID:
Note: The save() method automatically calls update_id() after successful create operations.

Auto-Generation Behavior

The UUID4 is generated in StashObject.__init__():
When UUID4 is Generated:
  • ✅ No id parameter provided: Scene(title="Test")
  • id=None explicitly passed: Scene(id=None, title="Test")
  • id with any string value: Scene(id="123", title="Test")

UNSET Sentinel Pattern (Three-Level Field System)

The Problem (Before)

Traditional two-level field systems only have:
  1. Set to a value: field = "value"
  2. Set to null: field = None
This makes partial updates impossible without sending all fields:

The Solution (After)

The UNSET sentinel provides a third state for “never touched”:

UNSET Sentinel Implementation

The UNSET sentinel is a singleton instance defined in types/unset.py:

Field Definition Pattern

Entity types should define fields with UNSET as the default:
Type Annotation Pattern:
  • For required fields: field: Type | UnsetType = UNSET
  • For optional fields: field: Type | None | UnsetType = UNSET
  • For always-required fields: field: Type (no default)

Using UNSET in to_input() Methods

When converting to GraphQL input types, exclude UNSET fields:
Note: The base StashObject.to_input() method handles this automatically when using Pydantic’s exclude_none=True. For full UNSET support, entity types need custom serialization logic.

Checking for UNSET

Use identity comparison (is) to check for UNSET:

Field Tracking with _received_fields

The Problem

When loading partial GraphQL fragments, we need to know which fields were actually included in the response:

The Solution

The _received_fields attribute tracks which fields were actually present in the GraphQL response:

How _received_fields Works

  1. Set during from_graphql(): When data comes from GraphQL, the _identity_map_validator tracks field names
  2. Merged on cache hits: If cached object receives new fields, they’re merged into existing _received_fields
  3. Empty for manual construction: Objects created with constructors have empty _received_fields

Use Cases for _received_fields

1. Detecting partial loads:
2. Merging partial fragments:
3. Debugging GraphQL queries:

Dirty Tracking with _snapshot

The Problem

When updating existing objects, we only want to send changed fields to avoid overwriting server data:

The Solution

The _snapshot attribute stores the original state after object construction. Dirty tracking methods compare current state to snapshot:

Dirty Tracking Methods

is_dirty() -> bool

Check if object has unsaved changes:

get_changed_fields() -> dict[str, Any]

Get dictionary of changed fields and their current values:
Note: Only fields in __tracked_fields__ are included in change detection.

mark_clean() -> None

Mark object as clean (no unsaved changes). Updates snapshot to current state:

mark_dirty() -> None

Force object to be considered dirty by clearing the snapshot:

How _snapshot Works

  1. Created in model_post_init(): After Pydantic initializes all fields, snapshot is taken
  2. Uses model_dump(): Leverages Pydantic’s serialization for accurate state capture
  3. Updated on mark_clean(): Save operations call mark_clean() to update snapshot
  4. Compared by get_changed_fields(): Compares current model_dump() to _snapshot

How to_input() Uses UNSET, _received_fields, and _snapshot

The Complete Flow

The to_input() method combines all three tracking mechanisms to generate minimal GraphQL mutation inputs:

New Objects: _to_input_all()

For new objects (UUID id, _is_new=True), include all fields that are not UNSET:
Behavior:
  • Processes all __field_conversions__ fields
  • Processes all __relationships__ fields
  • Excludes UNSET fields (never set)
  • Includes None fields (explicitly set to null)
  • Uses __create_input_type__ for validation

Existing Objects: _to_input_dirty()

For existing objects, only include changed fields based on snapshot comparison:
Behavior:
  1. Get changed fields: dirty_fields = set(self.get_changed_fields().keys())
  2. Process only dirty fields from __field_conversions__
  3. Process only dirty relationships from __relationships__
  4. Always include ID (required for updates)
  5. Exclude UNSET fields (unchanged or never loaded)
  6. Include None fields if dirty (changed to null)
  7. Uses __update_input_type__ for validation

Example: All Three Systems Together

Decision Matrix for to_input()

Key insight: UNSET exclusion happens at field processing level, dirty detection happens at change tracking level.

Practical Examples

Example 1: Creating a New Scene

Example 2: Partial Update (Only Changed Fields)

Example 3: Setting Field to Null vs Unsetting

Example 4: Checking Field State

Migration Guide for Entity Types

When migrating entity types to use the UNSET pattern:

Step 1: Import UNSET

Step 2: Update Field Definitions

Before:
After:

Step 3: Update to_input() Method

Add UNSET checks when converting to input:

Step 4: Update Tests

Test all three states:

Implementation Notes

Identity Map Compatibility

The UNSET pattern is compatible with the identity map (entity cache):

Performance Considerations

  • UUID4 generation: Minimal overhead (~0.1μs per object)
  • UNSET checks: Identity comparison (is) is O(1)
  • Memory: UNSET is a singleton, so only one instance exists in memory

Type Checking with mypy

The UNSET pattern is fully type-safe with mypy:

Testing Patterns

Test UUID4 Generation

Test UNSET Pattern

Future Enhancements

Pydantic v2 Serialization

When fully migrated to Pydantic v2, we can use custom serializers:

msgspec Migration

For msgspec migration, UNSET integrates with dec_hook:

Summary

UUID4 Auto-Generation

  • ✅ New objects get UUID4 automatically
  • is_new() checks if object has temporary ID
  • update_id() replaces UUID with server ID
  • save() handles ID updates automatically
  • _is_new attribute tracks new vs existing objects

UNSET Sentinel Pattern

  • ✅ Three-level field system: value, null, UNSET
  • ✅ Partial updates only send changed fields
  • ✅ Type-safe with mypy
  • ✅ Compatible with identity map caching
  • ✅ Minimal performance overhead
  • ✅ Distinguishes “not set” from “set to null”

Field Tracking with _received_fields

  • ✅ Tracks which fields came from GraphQL responses
  • ✅ Set automatically by from_graphql()
  • ✅ Merged on identity map cache hits
  • ✅ Empty for manually constructed objects
  • ✅ Enables partial fragment detection
  • ✅ Supports progressive field loading

Dirty Tracking with _snapshot

  • ✅ Stores original state after construction
  • is_dirty() detects any unsaved changes
  • get_changed_fields() returns modified fields
  • mark_clean() updates snapshot after save
  • mark_dirty() forces dirty state
  • ✅ Enables minimal update payloads

to_input() Integration

  • _to_input_all() for new objects (all non-UNSET fields)
  • _to_input_dirty() for existing objects (only changed fields)
  • ✅ Combines UNSET filtering with dirty detection
  • ✅ Always includes ID for updates
  • ✅ Respects both snapshot changes and UNSET exclusions

Best Practices

  1. Always use is for UNSET checks: if field is UNSET:
  2. Set UNSET as default: field: Type | UnsetType = UNSET
  3. Use from_graphql() for GraphQL data: Enables _received_fields tracking
  4. Check is_dirty() before save: Avoid unnecessary mutations
  5. Use get_changed_fields() for debugging: See exactly what changed
  6. Let UUID4 auto-generate: Don’t manually set for new objects
  7. Test all three field states: value, None, UNSET
  8. Trust the snapshot: mark_clean() called automatically by save()

See Also