> ## Documentation Index
> Fetch the complete documentation index at: https://docs.jakan.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Pydantic v2 Internals: How StashObject Works Around BaseModel

> Two significant Pydantic v2 behaviors that required workarounds in StashObject — private attribute stability and recursive repr blowup.

This document explains two significant Pydantic v2 behaviors that required workarounds in `StashObject`, and the architectural decisions behind each fix. Both stem from the same root: Pydantic v2's internal storage model has behaviors that are safe for typical models but dangerous for StashObject's identity map + bidirectional relationship pattern.

**Relevant versions**: Private attribute fix landed in v0.10.14. Shallow repr landed in v0.11.0b1.

***

## Background: Pydantic v2's Three Storage Dicts

Every Pydantic v2 `BaseModel` instance has three internal dictionaries:

| Dict                   | Purpose                             | Rebuilt by `validate_assignment`?           |
| ---------------------- | ----------------------------------- | ------------------------------------------- |
| `__dict__`             | Declared model fields               | **YES** — rebuilt on every field assignment |
| `__pydantic_extra__`   | Extra fields (when `extra="allow"`) | Partially (extra fields may migrate here)   |
| `__pydantic_private__` | `PrivateAttr` fields                | **NO** — stable across all operations       |

StashObject uses `validate_assignment=True` (to run validators on field updates) and `extra="allow"` (to accept unknown GraphQL fields without crashing). This combination creates the conditions for both issues below.

***

## Issue 1: Private Attributes Destroyed by `validate_assignment`

**Fixed in**: v0.10.14
**Location**: `stash_graphql_client/types/base.py`

### The Problem

StashObject maintains three private attributes for internal bookkeeping:

| Attribute          | Purpose                                                           |
| ------------------ | ----------------------------------------------------------------- |
| `_snapshot`        | Dirty tracking baseline — stores field values at last clean state |
| `_is_new`          | Tracks whether object needs `create` vs `update` mutation         |
| `_received_fields` | Tracks which fields came from GraphQL responses                   |

Originally, these were stored via `object.__setattr__()`, which writes directly to `__dict__`. This works in plain Python objects, but in Pydantic v2 with `validate_assignment=True`, every field assignment **rebuilds the entire `__dict__`** — silently destroying any non-field data stored there.

### How It Manifests

The bug only appears on identity map cache hits, not on fresh object construction:

```
1. Gallery(id="123") constructed         → _snapshot stored in __dict__ (all UNSET)
2. Identity map cache hit                → fields merged via setattr()
   └─ Each setattr triggers validate_assignment → __dict__ rebuilt repeatedly
   └─ _snapshot migrates to __pydantic_extra__ (stale copy)
3. mark_clean() called                   → new _snapshot stored in __dict__ (real values)
4. gallery.title = "New"                 → validate_assignment rebuilds __dict__
   └─ New _snapshot LOST
   └─ getattr falls through to __pydantic_extra__ → finds stale all-UNSET snapshot
5. gallery.is_dirty()                    → ALL fields appear dirty (real vs UNSET)
```

This caused phantom dirty fields — every `save()` call would send unnecessary GraphQL mutations for fields that hadn't actually changed.

### Minimal Reproduction (Pre-Fix)

```python theme={null}
from stash_graphql_client.types.gallery import Gallery

gallery = Gallery(id="123")
setattr(gallery, "title", "Real Title")    # Simulate identity map merge
setattr(gallery, "organized", False)

gallery.mark_clean()
snap_id = id(gallery._snapshot)

gallery.title = "New Title"                # Triggers validate_assignment
assert id(gallery._snapshot) != snap_id    # FAILS — snapshot was replaced!
# gallery._snapshot now points to the stale model_post_init snapshot (all UNSET)
```

### The Fix: Pydantic `PrivateAttr`

Convert all three attributes from `object.__setattr__()` storage to Pydantic `PrivateAttr` declarations:

```python theme={null}
from pydantic import PrivateAttr

class StashObject(BaseModel):
    # Stored in __pydantic_private__ — survives validate_assignment
    _snapshot: dict = PrivateAttr(default_factory=dict)
    _is_new: bool = PrivateAttr(default=False)
    _received_fields: set = PrivateAttr(default_factory=set)
```

`PrivateAttr` stores values in `__pydantic_private__`, which is **never** rebuilt by `validate_assignment`. All `object.__setattr__()` calls were replaced with direct assignment (`self._attr = value`), which Pydantic routes to `__pydantic_private__` automatically.

### Why It Only Appeared on Cache Hits

Objects constructed with all fields populated don't exhibit this bug because:

1. `model_post_init` creates `_snapshot` with real values (not UNSET)
2. Even if `__dict__` is rebuilt and `_snapshot` falls through to `__pydantic_extra__`, the stale copy also has real values
3. So `current_value == snapshot_value` holds true — dirty tracking appears to work

The bug only manifests when:

1. Object is constructed minimally (e.g., nested fragment with just `id`)
2. Fields are populated via `setattr()` (identity map merge path)
3. `model_post_init` snapshot has UNSET for all fields
4. `mark_clean()` creates new snapshot with real values
5. Next field assignment rebuilds `__dict__`, losing the snapshot
6. Fallback finds the stale all-UNSET snapshot from step 3

This is the exact sequence that occurs in production when `find_scenes()` or `find_galleries()` returns objects already in the identity map cache.

### Key Lesson

> **Never use `object.__setattr__()` to store state on Pydantic v2 models with `validate_assignment=True`.** Use `PrivateAttr` instead — it uses `__pydantic_private__`, the only dict that is guaranteed stable across all Pydantic operations.

***

## Issue 2: Recursive `__repr__` Exponential Blowup

**Fixed in**: v0.11.0b1
**Location**: `stash_graphql_client/types/base.py`, all entity type files

### The Problem

Pydantic v2's default `BaseModel.__repr__()` recursively renders **every field** of every nested model. With StashObject's bidirectional relationships, this walks the entire object graph:

```
Gallery.performers → [Performer] → .scenes → [Scene] → .performers → [Performer] ...
                                  → .groups → [Group] → ...
Gallery.scenes     → [Scene]     → .galleries → [Gallery] → ...
```

Pydantic's cycle detection prevents infinite recursion, but the traversal fans out exponentially across non-cyclic paths.

### Real-World Impact

A downstream consumer called `repr()` on changed field values before saving. On a dataset with \~3,000 posts (\~1,200 scenes, \~1,800 galleries):

| Metric                    | Value                                                                              |
| ------------------------- | ---------------------------------------------------------------------------------- |
| Per-gallery `repr()` cost | \~2.2 seconds (for `performers` field alone)                                       |
| Cost growth               | Worsens as identity map accumulates entities                                       |
| Total overhead            | **\~110 minutes** of CPU time generating strings immediately truncated to 80 chars |
| Memory                    | Multi-MB transient string allocations per call, contributing to OOM kills          |

Any consumer calling `repr()` — logging, debugging, REPL, pytest assertion output — hits the same problem.

### The Fix: Two-Tier Shallow Repr

StashObject now has two repr methods:

#### `_short_repr()` — Compact Nested Representation

Used when an object appears *inside* another object's repr. Collects all set+non-None fields from the `__short_repr_fields__` tuple:

```python theme={null}
def _short_repr(self) -> str:
    parts: list[str] = []
    for field in self.__short_repr_fields__:
        val = getattr(self, field, UNSET)
        if is_set(val) and val is not None:
            parts.append(f"{field}={val!r}")
    if not parts:
        return f"{self.__type_name__}(id={self.id!r})"
    return f"{self.__type_name__}({', '.join(parts)})"
```

If no fields match, falls back to `TypeName(id='...')`. Multi-field tuples show all matching fields — e.g., `__short_repr_fields__ = ("id", "name")` produces `Performer(id='123', name='Jane')`.

Each entity subclass declares which fields to include:

| Type      | `__short_repr_fields__` | Example                          |
| --------- | ----------------------- | -------------------------------- |
| Performer | `("name",)`             | `Performer(name='Jane')`         |
| Scene     | `("title",)`            | `Scene(title='Test Scene')`      |
| Tag       | `("name",)`             | `Tag(name='tag 1')`              |
| Studio    | `("name",)`             | `Studio(name='Acme')`            |
| Gallery   | `("title",)`            | `Gallery(title='2024 Post')`     |
| Group     | `("name",)`             | `Group(name='Series 1')`         |
| Image     | `("title",)`            | `Image(title='Photo')`           |
| VideoFile | `("basename",)`         | `VideoFile(basename='clip.mp4')` |
| Folder    | `("path",)`             | `Folder(path='/media/videos')`   |

Fallback: if label field is UNSET or None, falls back to `TypeName(id='123')`.

#### `__repr__()` — Full Representation

Shows **all non-UNSET model fields** (not just tracked fields), with relationship fields rendered shallowly:

```python theme={null}
def __repr__(self) -> str:
    parts: list[str] = [f"id={self.id!r}"]

    for field_name in sorted(self.__class__.model_fields):
        if field_name == "id":
            continue
        val = getattr(self, field_name, UNSET)
        if not is_set(val):
            continue

        if isinstance(val, StashObject):
            parts.append(f"{field_name}={val._short_repr()}")
        elif isinstance(val, list) and val and isinstance(val[0], StashObject):
            limit = self._SHORT_REPR_LIST_LIMIT  # default: 2
            items = [item._short_repr() for item in val[:limit]]
            suffix = f", ..{len(val) - limit} more" if len(val) > limit else ""
            parts.append(f"{field_name}=[{', '.join(items)}{suffix}]")
        else:
            r = repr(val)
            if len(r) > 200:
                r = r[:197] + "..."
            parts.append(f"{field_name}={r}")

    return f"{self.__type_name__}({', '.join(parts)})"
```

### Example Output

**Before** (Pydantic default — truncated, actual output is megabytes):

```
Performer(id='abc123', name='Jane', alias_list=['JD'], tags=[Tag(id='t1', name='blonde',
parents=[Tag(id='t2', name='hair', parents=[...], children=[Tag(id='t1', ...)])],
children=[...])], scenes=[Scene(id='s1', title='...', performers=[Performer(id='abc123',
name='Jane', alias_list=['JD'], tags=[...], scenes=[Scene(id='s2', ...
... (continues for megabytes)
```

**After** (shallow repr):

```
Performer(id='abc123', alias_list=['JD'], favorite=False, name='Jane',
  scenes=[Scene(title='S1'), Scene(title='S2'), ..1198 more],
  tags=[Tag(name='blonde'), Tag(name='brunette'), ..3 more])
```

### Design Decisions

**Why show all model fields, not just tracked fields?**

The initial proposal showed only `__tracked_fields__`, but the final implementation shows all non-UNSET model fields. This provides more complete debugging information — fields like `rating100`, `scene_count`, and `date` are useful in repr output even though they aren't tracked for dirty checking.

**Why `sorted()` on field names?**

Deterministic output across runs. Without sorting, dict iteration order could vary, making log diffs and test assertions unreliable.

**Why `self.__class__.model_fields` instead of `self.model_fields`?**

Pydantic v2.11 deprecated instance-level `model_fields` access. Using the class-level accessor avoids deprecation warnings.

**Why no dirty indicator (`*` suffix)?**

The original proposal included `"*"` when `is_dirty()` returns True. This was removed because `is_dirty()` does field-by-field comparison including list traversal — too much work for a `__repr__` method that might be called frequently in logging or debugging contexts. A repr should be fast and side-effect-free.

**Why not use `__repr_args__` (Pydantic's hook)?**

Pydantic v2's `__repr_args__` returns `(field_name, value)` tuples that Pydantic then `repr()`s recursively. There's no way to control relationship rendering at that level — the only solution is to override `__repr__` entirely.

**Why truncate scalars at 200 characters (not 60)?**

Fields like `details` can contain multi-paragraph text. The initial proposal used 60 chars, but this was too aggressive — URLs, file paths, and aliases are commonly 80-150 chars. 200 chars provides a better balance.

**Why first 2 items in list repr (not count-only)?**

Showing `tags=[5 Tag]` is compact but loses information. Showing `tags=[Tag(name='blonde'), Tag(name='brunette'), ..3 more]` gives immediate identification of the first two items, which is usually enough to understand the relationship contents without expanding the full list.

**What about `__str__`?**

Left as Pydantic's default (which delegates to `__repr__`). This means `str(obj)`, `f"{obj}"`, and `print(obj)` all use the shallow repr — there's no use case where a consumer wants the multi-MB recursive output.

***

## The Common Thread

Both issues stem from Pydantic v2's `BaseModel` being designed for simple data containers, not for objects with:

* **Identity map caching** — objects are mutated after construction via `setattr()`
* **Bidirectional relationships** — objects reference each other (Scene→Performer→Scene)
* **Internal bookkeeping state** — private attributes that must survive field mutations

StashObject works within Pydantic v2's constraints by:

1. Using `__pydantic_private__` (via `PrivateAttr`) for internal state that must be stable
2. Overriding `__repr__` to prevent recursive traversal of the relationship graph
3. Using `__class__.model_fields` for introspection to avoid deprecated instance access
4. Keeping `validate_assignment=True` for field validation while protecting private state from its dict-rebuilding side effect