# Migration Refactor Plan ## Overview Replace the current `StorageKey.migrationSources` approach with a protocol-based migration system that supports conditional logic, data transformation, and complex migration scenarios. ### Design Guardrails (Align with Current Patterns) - Keep `StorageKey` as the single source of truth for key metadata. - Use type erasure (`AnyStorageKey`-style) for heterogeneous migrations. - Avoid `any StorageKey` existentials in public APIs; prefer `AnyStorageKey` and `StorageKeyDescriptor`. - Migrations must use the router instance passed in (no `StorageRouter.shared` inside migration logic). ### File Organization Follow AGENTS.md Clean Architecture and File Organization Principles for all new code: - **One Public Type Per File**: Each file contains exactly one public struct/class/enum (private helpers OK if small). - **Keep Files Lean (<300 Lines)**: Extract complex logic to dedicated types if needed (e.g., separate files for each concrete migration). - **No Duplicate Code**: Search for existing patterns before writing (e.g., reuse AnyStorageKey patterns). - **Logical Grouping**: Organize by feature, not type: ``` Sources/LocalData/ ├── Migrations/ # New folder for concrete migrations │ ├── SimpleLegacyMigration.swift │ ├── TransformingMigration.swift │ └── AggregatingMigration.swift ├── Models/ # Data models │ ├── AnyStorageMigration.swift │ ├── MigrationResult.swift │ └── MigrationContext.swift ├── Protocols/ # Interfaces │ ├── StorageMigration.swift │ ├── TransformingMigration.swift │ └── AggregatingMigration.swift └── Utilities/ # Helpers ├── DeviceInfo.swift └── SystemInfo.swift ``` - Ensure documentation files (e.g., Migration.md) stay in sync with changes. ## Current System Analysis ### Limitations of Current System - Strict 1:Many legacy-to-new key migration (last one wins) - No conditional logic based on content or state - No data transformation capabilities during migration - Cannot aggregate multiple sources - Hardcoded migration logic in StorageRouter - Limited error handling and recovery options ### Strengths to Preserve - Simple API for basic migrations - Automatic fallback on `get()` operations - Manual migration sweep support - Integration with storage catalog system - Comprehensive test coverage ## Phase 1: Core Protocol Infrastructure ### 1.1 New Protocol Files #### `Sources/LocalData/Protocols/StorageMigration.swift` ```swift /// Core migration protocol with high-level methods public protocol StorageMigration: Sendable { associatedtype DestinationKey: StorageKey /// The destination storage key where migrated data will be stored var destinationKey: DestinationKey { get } /// Validate if migration should proceed (conditional logic) func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool /// Execute the migration process func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult } ``` #### `Sources/LocalData/Models/AnyStorageMigration.swift` ```swift /// Type-erased wrapper for StorageMigration to match AnyStorageKey patterns public struct AnyStorageMigration: Sendable { public let destinationDescriptor: StorageKeyDescriptor private let shouldMigrateAction: @Sendable (StorageRouter, MigrationContext) async throws -> Bool private let migrateAction: @Sendable (StorageRouter, MigrationContext) async throws -> MigrationResult public init(_ migration: M) { self.destinationDescriptor = .from(migration.destinationKey) self.shouldMigrateAction = migration.shouldMigrate self.migrateAction = migration.migrate } public func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { try await shouldMigrateAction(router, context) } public func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { try await migrateAction(router, context) } } ``` #### `Sources/LocalData/Models/MigrationResult.swift` ```swift /// Result of a migration operation with detailed information public struct MigrationResult: Sendable { public let success: Bool public let migratedCount: Int public let errors: [MigrationError] public let metadata: [String: AnyCodable] public let duration: TimeInterval public init(success: Bool, migratedCount: Int = 0, errors: [MigrationError] = [], metadata: [String: AnyCodable] = [:], duration: TimeInterval = 0) { self.success = success self.migratedCount = migratedCount self.errors = errors self.metadata = metadata self.duration = duration } } /// Migration-specific error types public enum MigrationError: Error, Sendable, Equatable { case validationFailed(String) case transformationFailed(String) case storageFailed(StorageError) case conditionalMigrationFailed case migrationInProgress case sourceDataNotFound case incompatibleTypes(String) case aggregationFailed(String) public var localizedDescription: String { switch self { case .validationFailed(let message): return "Migration validation failed: \(message)" case .transformationFailed(let message): return "Data transformation failed: \(message)" case .storageFailed(let storageError): return "Storage operation failed: \(storageError.localizedDescription)" case .conditionalMigrationFailed: return "Conditional migration criteria not met" case .migrationInProgress: return "Migration already in progress" case .sourceDataNotFound: return "No data found in migration source" case .incompatibleTypes(let message): return "Type incompatibility: \(message)" case .aggregationFailed(let message): return "Data aggregation failed: \(message)" } } } ``` #### `Sources/LocalData/Models/MigrationContext.swift` ```swift /// Context information available for conditional migrations public struct MigrationContext: Sendable { public let appVersion: String public let deviceInfo: DeviceInfo public let migrationHistory: [String: Date] public let userPreferences: [String: AnyCodable] public let systemInfo: SystemInfo public init( appVersion: String = Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "Unknown", deviceInfo: DeviceInfo = .current, migrationHistory: [String: Date] = [:], userPreferences: [String: AnyCodable] = [:], systemInfo: SystemInfo = .current ) { self.appVersion = appVersion self.deviceInfo = deviceInfo self.migrationHistory = migrationHistory self.userPreferences = userPreferences self.systemInfo = systemInfo } } ``` ### 1.2 Update StorageKey Protocol #### Modify `Sources/LocalData/Protocols/StorageKey.swift` ```swift public protocol StorageKey: Sendable, CustomStringConvertible { associatedtype Value: Codable & Sendable var name: String { get } var domain: StorageDomain { get } var security: SecurityPolicy { get } var serializer: Serializer { get } var owner: String { get } var availability: PlatformAvailability { get } var syncPolicy: SyncPolicy { get } /// Optional migration for simple or complex scenarios var migration: AnyStorageMigration? { get } } extension StorageKey { public var migration: AnyStorageMigration? { nil } } ``` ### 1.3 Update AnyStorageKey #### Modify `Sources/LocalData/Models/AnyStorageKey.swift` ```swift public struct AnyStorageKey: Sendable { public internal(set) var descriptor: StorageKeyDescriptor public internal(set) var migration: AnyStorageMigration? private let migrateAction: @Sendable (StorageRouter) async throws -> Void public init(_ key: Key) { self.descriptor = .from(key) self.migration = key.migration self.migrateAction = { router in _ = try await router.forceMigration(for: key) } } // ... existing initializers/withCatalog/migrate(on:) ... } ``` ## Phase 2: Concrete Migration Implementations ### 2.1 Legacy Support Migration #### `Sources/LocalData/Migrations/SimpleLegacyMigration.swift` ```swift /// Simple 1:1 legacy migration public struct SimpleLegacyMigration: StorageMigration { public typealias DestinationKey = Destination public let destinationKey: Destination public let sourceKey: AnyStorageKey public init(destinationKey: Destination, sourceKey: AnyStorageKey) { self.destinationKey = destinationKey self.sourceKey = sourceKey } public func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { // Check if destination is empty and legacy source has data try await router.exists(descriptor: sourceKey.descriptor) } public func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { let startTime = Date() var errors: [MigrationError] = [] var metadata: [String: AnyCodable] = [:] var migratedCount = 0 do { if let sourceData = try await router.retrieve(for: sourceKey.descriptor) { let unsecuredData = try await router.applySecurity(sourceData, for: sourceKey.descriptor, isEncrypt: false) let value = try router.deserialize(unsecuredData, with: destinationKey.serializer) // Store using destination key configuration try await router.set(value, for: destinationKey) // Delete source try await router.delete(for: sourceKey.descriptor) migratedCount = 1 Logger.info("!!! [MIGRATION] Successfully migrated from '\(sourceKey.descriptor.name)' to '\(destinationKey.descriptor.name)'") } } catch { errors.append(.storageFailed(error as? StorageError ?? .unknown(error))) Logger.error("!!! [MIGRATION] Failed to migrate from '\(sourceKey.descriptor.name)': \(error)") } let duration = Date().timeIntervalSince(startTime) return MigrationResult( success: errors.isEmpty, migratedCount: migratedCount, errors: errors, metadata: metadata, duration: duration ) } } ``` ### 2.2 Conditional Migration #### `Sources/LocalData/Migrations/AppVersionConditionalMigration.swift` ```swift /// Conditional migration for app version-based migration public struct AppVersionConditionalMigration: ConditionalMigration { public typealias DestinationKey = Destination public let destinationKey: Destination public let minAppVersion: String public let fallbackMigration: AnyStorageMigration public init(destinationKey: Destination, minAppVersion: String, fallbackMigration: AnyStorageMigration) { self.destinationKey = destinationKey self.minAppVersion = minAppVersion self.fallbackMigration = fallbackMigration } public func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { return context.appVersion.compare(minAppVersion, options: .numeric) == .orderedAscending } public func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Use fallback migration if condition is met return try await fallbackMigration.migrate(using: router, context: context) } } ``` ### 2.3 Transforming Migration #### `Sources/LocalData/Migrations/TransformingMigration.swift` ```swift /// Migration protocol that supports data transformation during migration public protocol TransformingMigration: StorageMigration { associatedtype SourceKey: StorageKey var sourceKey: SourceKey { get } func transform(_ source: SourceKey.Value) async throws -> DestinationKey.Value } public struct DefaultTransformingMigration: TransformingMigration { public typealias SourceKey = Source public typealias DestinationKey = Destination public let destinationKey: Destination public let sourceKey: Source public let transform: (Source.Value) async throws -> Destination.Value public init( destinationKey: Destination, sourceKey: Source, transform: @escaping (Source.Value) async throws -> Destination.Value ) { self.destinationKey = destinationKey self.sourceKey = sourceKey self.transform = transform } public func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { try await router.exists(sourceKey) } public func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { let startTime = Date() do { // Get source data let sourceData = try await router.get(sourceKey) // Transform data let transformedData = try await transform(sourceData) // Store transformed data try await router.set(transformedData, for: destinationKey) // Delete source try await router.remove(sourceKey) let duration = Date().timeIntervalSince(startTime) return MigrationResult(success: true, migratedCount: 1, duration: duration) } catch { let duration = Date().timeIntervalSince(startTime) return MigrationResult( success: false, errors: [.transformationFailed(error.localizedDescription)], duration: duration ) } } } ``` ### 2.4 Aggregating Migration #### `Sources/LocalData/Migrations/AggregatingMigration.swift` ```swift /// Migration protocol that combines multiple sources into single destination public protocol AggregatingMigration: StorageMigration { var sourceKeys: [AnyStorageKey] { get } func aggregate(_ sources: [AnyCodable]) async throws -> DestinationKey.Value } public struct DefaultAggregatingMigration: AggregatingMigration { public typealias DestinationKey = Destination public let destinationKey: Destination public let sourceKeys: [AnyStorageKey] public let aggregate: ([AnyCodable]) async throws -> Destination.Value public init( destinationKey: Destination, sourceKeys: [AnyStorageKey], aggregate: @escaping ([AnyCodable]) async throws -> Destination.Value ) { self.destinationKey = destinationKey self.sourceKeys = sourceKeys self.aggregate = aggregate } public func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { // Check if any source has data and destination is empty var hasSourceData = false for source in sourceKeys { if try await router.exists(descriptor: source.descriptor) { hasSourceData = true break } } let destinationExists = try await router.exists(destinationKey) return hasSourceData && !destinationExists } public func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { let startTime = Date() var sourceData: [AnyCodable] = [] var migratedCount = 0 do { // Collect data from all sources for source in sourceKeys { if let data = try await router.retrieve(for: source.descriptor) { let unsecuredData = try await router.applySecurity(data, for: source.descriptor, isEncrypt: false) if let codable = try? JSONDecoder().decode(AnyCodable.self, from: unsecuredData) { sourceData.append(codable) migratedCount += 1 } } } // Aggregate data let aggregatedData = try await aggregate(sourceData) // Store aggregated data try await router.set(aggregatedData, for: destinationKey) // Delete source data for source in sourceKeys { try? await router.delete(for: source.descriptor) } let duration = Date().timeIntervalSince(startTime) return MigrationResult(success: true, migratedCount: migratedCount, duration: duration) } catch { let duration = Date().timeIntervalSince(startTime) return MigrationResult( success: false, errors: [.aggregationFailed(error.localizedDescription)], duration: duration ) } } } ``` ## Phase 3: StorageRouter Integration ### WatchOS and Sync Integration Migrations respect key.availability and syncPolicy. Provide default implementations in StorageMigration extension to handle platform checks and post-migration sync automatically, minimizing developer complexity. #### Example Default Extension ```swift extension StorageMigration { public func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { guard destinationKey.availability.isAvailable(on: context.deviceInfo.platform) else { return false } if destinationKey.syncPolicy != .never { guard await WatchConnectivityService.shared.isLeadDevice else { return false } } return true // Override for custom logic } public func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Core logic... if destinationKey.syncPolicy != .never { await WatchConnectivityService.shared.syncKey(destinationKey) } return result } } ``` ### 3.1 Update StorageRouter #### Modify `Sources/LocalData/Services/StorageRouter.swift` **New Properties:** ```swift public actor StorageRouter: StorageProviding { // ... existing properties ... private var migrationHistory: [String: Date] = [:] // No initializer changes required } ``` **Internal Migration Helpers (Visibility Updates):** ```swift // Promote existing private helpers to internal for migration support. internal func retrieve(for descriptor: StorageKeyDescriptor) async throws -> Data? { ... } internal func delete(for descriptor: StorageKeyDescriptor) async throws { ... } internal func deserialize(_ data: Data, with serializer: Serializer) throws -> Value { ... } // New helper for type-erased existence checks. internal func exists(descriptor: StorageKeyDescriptor) async throws -> Bool { ... } // Build a single context instance for migration decisions. internal func buildMigrationContext() async -> MigrationContext { MigrationContext(migrationHistory: migrationHistory) } internal func recordMigration(for descriptor: StorageKeyDescriptor) { migrationHistory[descriptor.name] = Date() } ``` **Updated Methods:** **Enhanced `get` method:** ```swift public func get(_ key: Key) async throws -> Key.Value { try validateCatalogRegistration(for: key) try validatePlatformAvailability(for: key) // 1. Check primary location if let securedData = try await retrieve(for: .from(key)) { let data = try await applySecurity(securedData, for: .from(key), isEncrypt: false) let result = try deserialize(data, with: key.serializer) Logger.debug("<<< [STORAGE] GET SUCCESS: \(key.name)") return result } // 2. NEW: Try protocol-based migration if defined if let migration = key.migration { let context = await buildMigrationContext() if try await migration.shouldMigrate(using: self, context: context) { Logger.info("!!! [MIGRATION] Starting migration for key: \(key.name)") let result = try await advanced.migrate(using: self, context: context) if result.success, let finalData = try await retrieve(for: .from(key)) { let unsecuredData = try await applySecurity(finalData, for: .from(key), isEncrypt: false) let value = try deserialize(unsecuredData, with: key.serializer) recordMigration(for: .from(key)) return value } } } Logger.debug("<<< [STORAGE] GET NOT FOUND: \(key.name)") throw StorageError.notFound } ``` **Update `migrateAllRegisteredKeys`:** ```swift private func migrateAllRegisteredKeys() async throws { for entry in registeredKeys.values { guard entry.migration != nil else { continue } try await entry.migrate(on: self) } } ``` **New Methods:** ```swift /// Get migration history for a key public func migrationHistory(for key: Key) async -> Date? { migrationHistory[key.name] } /// Force migration for a specific key using its attached migration public func forceMigration(for key: Key) async throws -> MigrationResult { guard let migration = key.migration else { throw MigrationError.sourceDataNotFound } Logger.debug(">>> [MIGRATION] FORCED MIGRATION: \(key.name)") let context = await buildMigrationContext() let result = try await migration.migrate(using: self, context: context) if result.success { recordMigration(for: .from(key)) } return result } ``` **Catalog Registration Ordering (Multi-Catalog Safe):** - When `migrateImmediately` is true, only attempt migrations for keys that define `migration`. - If multiple catalogs attempt to register migrations for the same key name, treat it as a duplicate registration and throw, matching existing duplicate key behavior. ## Phase 4: Sample App Updates ### 4.1 Update MigrationDemo View #### Modify `SecureStorageSample/Views/MigrationDemo.swift` **New Structure:** ```swift @MainActor struct MigrationDemo: View { @State private var selectedTab = 0 @State private var legacyValue = "" @State private var modernValue = "" @State private var conditionalValue = "" @State private var transformedValue = "" @State private var aggregatedValue = "" @State private var statusMessage = "" @State private var isLoading = false @State private var migrationResults: [MigrationResult] = [] var body: some View { TabView(selection: $selectedTab) { legacyMigrationView.tabItem { Label("Legacy", systemImage: "arrow.clock") } conditionalMigrationView.tabItem { Label("Conditional", systemImage: "questionmark.circle") } transformingMigrationView.tabItem { Label("Transform", systemImage: "arrow.triangle.2.circlepath") } aggregatingMigrationView.tabItem { Label("Aggregate", systemImage: "square.stack.3d.up") } } .navigationTitle("Data Migration") .navigationBarTitleDisplayMode(.inline) } // MARK: - Tab Views @ViewBuilder private var legacyMigrationView: some View { // ... existing migration demo UI ... } @ViewBuilder private var conditionalMigrationView: some View { Form { Section("Conditional Migration") { Text("Migrate data only if app version is older than 2.0, demonstrating conditional logic.") .font(.caption) .foregroundStyle(.secondary) } Section("Current App Version") { Text(Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "Unknown") .font(.title2) .bold() } Section("Test Conditional Migration") { Button(action: testConditionalMigration) { Label("Test Conditional Migration", systemImage: "play.circle") } .disabled(isLoading) if !conditionalValue.isEmpty { LabeledContent("Migrated Value", value: conditionalValue) .foregroundStyle(.green) .bold() } } } } @ViewBuilder private var transformingMigrationView: some View { Form { Section("Data Transformation") { Text("Transform old UserProfileV1 format to new UserProfileV2 format, demonstrating data structure changes.") .font(.caption) .foregroundStyle(.secondary) } Section("Legacy Profile") { TextField("First Name", text: $legacyFirstName) TextField("Last Name", text: $legacyLastName) Button(action: saveLegacyProfile) { Label("Save Legacy Profile", systemImage: "person.crop.circle") } .disabled(legacyFirstName.isEmpty || legacyLastName.isEmpty || isLoading) } Section("Transform and Migrate") { Button(action: testTransformingMigration) { Label("Transform and Migrate", systemImage: "arrow.triangle.2.circlepath") } .disabled(isLoading) if !transformedValue.isEmpty { Text(transformedValue) .font(.caption) .foregroundStyle(.green) } } } } @ViewBuilder private var aggregatingMigrationView: some View { Form { Section("Data Aggregation") { Text("Combine multiple legacy settings keys into single unified settings, demonstrating multi-source aggregation.") .font(.caption) .foregroundStyle(.secondary) } Section("Legacy Settings") { // Multiple settings fields for aggregation Toggle("Notifications", isOn: $notificationsEnabled) Toggle("Auto-Backup", isOn: $autoBackupEnabled) TextField("Theme", text: $selectedTheme) Button(action: saveLegacySettings) { Label("Save Legacy Settings", systemImage: "gear") } .disabled(isLoading) } Section("Aggregate and Migrate") { Button(action: testAggregatingMigration) { Label("Aggregate Settings", systemImage: "square.stack.3d.up") } .disabled(isLoading) if !aggregatedValue.isEmpty { Text(aggregatedValue) .font(.caption) .foregroundStyle(.green) } } } } } ``` ### 4.2 New Migration Implementations in Sample App #### New Files: `SecureStorageSample/Migrations/` **`ConditionalProfileMigration.swift`** ```swift import LocalData import SharedKit /// Conditional migration that only runs for old app versions struct ConditionalProfileMigration: ConditionalMigration { typealias DestinationKey = ModernConditionalProfileKey let destinationKey = ModernConditionalProfileKey() func shouldMigrate(context: MigrationContext) async throws -> Bool { // Only migrate if app version is older than 2.0 return context.appVersion.compare("2.0.0", options: .numeric) == .orderedAscending } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Use simple legacy migration if condition is met let legacyMigration = SimpleLegacyMigration( destinationKey: destinationKey, sourceKey: .key(LegacyConditionalProfileKey()) ) return try await legacyMigration.migrate(using: router, context: context) } } ``` **`ProfileTransformMigration.swift`** ```swift /// Transform legacy UserProfileV1 to modern UserProfileV2 struct ProfileTransformMigration: TransformingMigration { typealias SourceKey = LegacyUserProfileV1Key typealias DestinationKey = ModernUserProfileKey let destinationKey = ModernUserProfileKey() let sourceKey = LegacyUserProfileV1Key() func transform(_ source: UserProfileV1) async throws -> UserProfile { UserProfile( id: source.id, fullName: "\(source.firstName) \(source.lastName)", email: source.email, createdAt: source.createdAt, updatedAt: Date(), preferences: UserPreferences( notifications: true, theme: "system", language: "en" ) ) } } // Legacy profile format struct UserProfileV1: Codable { let id: UUID let firstName: String let lastName: String let email: String let createdAt: Date } ``` **`AggregatedSettingsMigration.swift`** ```swift /// Aggregate multiple settings sources into unified settings struct AggregatedSettingsMigration: AggregatingMigration { typealias DestinationKey = UnifiedSettingsKey let destinationKey = UnifiedSettingsKey() let sourceKeys: [AnyStorageKey] = [ .key(NotificationSettingsKey()), .key(BackupSettingsKey()), .key(ThemeSettingsKey()) ] func aggregate(_ sources: [AnyCodable]) async throws -> UnifiedSettings { var notifications = false var autoBackup = false var theme = "system" // Extract individual settings from sources for source in sources { if let data = source.value as? [String: Any] { if let notifs = data["notifications"] as? Bool { notifications = notifs } if let backup = data["autoBackup"] as? Bool { autoBackup = backup } if let selectedTheme = data["theme"] as? String { theme = selectedTheme } } } return UnifiedSettings( notifications: notifications, autoBackup: autoBackup, theme: theme, version: "2.0", lastUpdated: Date() ) } } struct UnifiedSettings: Codable { let notifications: Bool let autoBackup: Bool let theme: String let version: String let lastUpdated: Date } ``` ### 4.3 New Storage Keys for Sample App #### `SecureStorageSample/StorageKeys/Migration/` **`ConditionalMigrationKeys.swift`** ```swift extension StorageKeys { struct LegacyConditionalProfileKey: StorageKey { typealias Value = UserProfile let name = "legacy_conditional_profile" let domain: StorageDomain = .userDefaults(suite: nil) let security: SecurityPolicy = .none let serializer: Serializer = .json let owner = "MigrationDemo" let description = "Legacy profile for conditional migration demo." let availability: PlatformAvailability = .all let syncPolicy: SyncPolicy = .never } struct ModernConditionalProfileKey: StorageKey { typealias Value = UserProfile let name = "modern_conditional_profile" let domain: StorageDomain = .keychain(service: "com.mbrucedogs.securestorage") let security: SecurityPolicy = .keychain( accessibility: .afterFirstUnlock, accessControl: .userPresence ) let serializer: Serializer = .json let owner = "MigrationDemo" let description = "Modern profile for conditional migration demo." let availability: PlatformAvailability = .all let syncPolicy: SyncPolicy = .never var migration: AnyStorageMigration? { AnyStorageMigration(ConditionalProfileMigration()) } } } ``` ## Phase 6: Testing Implementation ### Audit and Update Existing Tests - Review and update existing MigrationTests.swift to remove references to migrationSources and test new migration patterns. - Add tests for failure scenarios, rollbacks, and watchOS-specific cases (e.g., skipped migrations on watch). ## Phase 7: Documentation Updates Move documentation updates to after testing completion. ### 7.1 Update Migration.md - Update the Data Migration section to describe migration for simple and complex cases. - Replace migrationSources examples with new patterns. - Update migrateImmediately description to handle both simple and advanced migrations. ## Phase 8: Additional Utility Files ### 5.1 Update Migration.md #### File: `Documentation/Migration.md` **Complete Rewrite:** ```markdown # LocalData Migration Guide ## Overview LocalData provides a flexible, protocol-based migration system that supports: - **Simple Legacy Migration**: Drop-in replacement for string-based key migration - **Conditional Migration**: Execute migration based on app version, device state, or custom logic - **Data Transformation**: Convert data structures and types during migration - **Data Aggregation**: Combine multiple sources into single destination ## Protocol-Based Migration (Recommended) ### Core Migration Protocol The `StorageMigration` protocol provides the foundation for all migration types: ```swift public protocol StorageMigration: Sendable { associatedtype DestinationKey: StorageKey var destinationKey: DestinationKey { get } func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult } ### Migration Types #### 1. Simple Legacy Migration Simple 1:1 migration for a legacy key: ```swift struct LegacyToModernMigration: StorageMigration { typealias DestinationKey = ModernUserProfileKey let destinationKey = ModernUserProfileKey() let sourceKey: AnyStorageKey func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { // Check if destination empty and legacy has data } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Move data from legacy to modern } } // Register with key extension ModernUserProfileKey { var migration: AnyStorageMigration? { AnyStorageMigration(SimpleLegacyMigration( destinationKey: self, sourceKey: .key(LegacyUserProfileKey()) )) } } ``` #### 2. Conditional Migration Execute migration based on custom logic: ```swift struct VersionBasedMigration: ConditionalMigration { typealias DestinationKey = ModernUserProfileKey let destinationKey = ModernUserProfileKey() func shouldMigrate(context: MigrationContext) async throws -> Bool { // Only migrate from app versions < 2.0 context.appVersion.compare("2.0.0", options: .numeric) == .orderedAscending } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Migration logic here } } ``` #### 3. Data Transformation Migration Convert data structures during migration: ```swift struct ProfileUpgradeMigration: TransformingMigration { typealias SourceKey = LegacyProfileKey typealias DestinationKey = ModernProfileKey let destinationKey = ModernProfileKey() let sourceKey = LegacyProfileKey() func transform(_ source: ProfileV1) async throws -> ProfileV2 { ProfileV2( id: source.id, fullName: "\(source.firstName) \(source.lastName)", email: source.email, preferences: UserPreferences.fromLegacy(source.preferences) ) } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Get source, transform, store, cleanup } } ``` #### 4. Data Aggregation Migration Combine multiple sources: ```swift struct SettingsAggregationMigration: AggregatingMigration { typealias DestinationKey = UnifiedSettingsKey let destinationKey = UnifiedSettingsKey() let sourceKeys: [AnyStorageKey] = [ .key(NotificationSettingsKey()), .key(ThemeSettingsKey()), .key(PrivacySettingsKey()) ] func aggregate(_ sources: [AnyCodable]) async throws -> UnifiedSettings { // Combine all settings into unified structure } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { // Collect from all sources, aggregate, store, cleanup } } ``` ### Migration History Migration history is tracked in-memory by `StorageRouter` to support conditional logic: ```swift // Get last migration date let lastMigrationDate = await StorageRouter.shared.migrationHistory(for: key) // Force migration let result = try await StorageRouter.shared.forceMigration(for: key) ``` ### 5.2 Update README.md (Post-Implementation) #### File: `README.md` - Update the Data Migration section to describe the new `StorageKey.migration` flow. - Replace `migrationSources` examples with `AnyStorageMigration` examples. - Update `migrateImmediately` description to reflect migrations attached to keys. ### 5.3 Update Proposal.md (Post-Implementation) #### File: `Proposal.md` - Reflect the new protocol-based migration approach in the proposal narrative. - Remove legacy migration references (`migrationSources`) from the proposal. ### Migration Results All migrations return a `MigrationResult`: ```swift public struct MigrationResult { public let success: Bool public let migratedCount: Int public let errors: [MigrationError] public let metadata: [String: AnyCodable] public let duration: TimeInterval } ``` ### Error Handling Comprehensive error types for migration failures: ```swift public enum MigrationError: Error, Sendable { case validationFailed(String) case transformationFailed(String) case storageFailed(StorageError) case conditionalMigrationFailed case sourceDataNotFound // ... more cases } ``` ## Migration Strategies ### Automatic Migration Migrations trigger automatically when accessing keys with no data: 1. Check primary location 2. Execute `shouldMigrate(using:context:)` if migration protocol exists 3. Run migration if condition is met 4. Return migrated data ### Manual Migration Force migration with `forceMigration(for:)`: ```swift let result = try await StorageRouter.shared.forceMigration(for: key) if result.success { print("Migrated \(result.migratedCount) items in \(result.duration)s") } ``` ### Proactive Migration (Sweep) Migrate all registered keys at app launch: ```swift try await StorageRouter.shared.registerCatalog(AppStorageCatalog.self, migrateImmediately: true) ``` ## Best Practices 1. **Use Protocol-Based Migration**: More flexible and powerful than legacy arrays 2. **Implement Conditional Logic**: Avoid unnecessary migrations 3. **Transform Data Early**: Migrate to modern data structures 4. **Test Migrations**: Comprehensive testing for all scenarios 5. **Handle Errors**: Provide fallbacks and recovery options 6. **Document Migrations**: Clear descriptions for audit reports ## Testing Migrations Use the new protocol system for comprehensive testing: ```swift @Test func conditionalMigrationTest() async throws { let migration = VersionBasedMigration() let context = MigrationContext() let shouldMigrate = try await migration.shouldMigrate(using: router, context: context) #expect(shouldMigrate == true) let result = try await migration.migrate(using: router, context: context) #expect(result.success == true) } ``` ## Migration History The system tracks migration history to prevent re-migration: `MigrationContext` is built by `StorageRouter` when running migrations and includes in-memory migration history. In tests, you can create a context manually with a custom history to validate conditional behavior. ## Performance Considerations - **Conditional Logic**: Fast checks before expensive operations - **Batch Operations**: Efficient processing of multiple sources - **Error Recovery**: Capture errors in `MigrationResult`; rollback is out of scope for Phase 1 - **Progress Reporting**: Defer until a concrete UI/reporting requirement exists ``` ### 5.2 Update README.md #### File: `README.md` **Add to Usage Section:** ```markdown ## Usage ### 1. Define Keys Extend `StorageKeys` with your own key types: ```swift import LocalData extension StorageKeys { struct UserTokenKey: StorageKey { typealias Value = String let name = "user_token" let domain: StorageDomain = .keychain(service: "com.myapp") let security: SecurityPolicy = .keychain( accessibility: .afterFirstUnlock, accessControl: .biometryAny ) let serializer: Serializer = .json let owner = "AuthService" let description = "Stores the current user auth token." let availability: PlatformAvailability = .phoneOnly let syncPolicy: SyncPolicy = .never } } ``` ### 2. Protocol-Based Migration (Recommended) For complex migration scenarios, implement the `StorageMigration` protocol: ```swift struct TokenMigration: StorageMigration { typealias DestinationKey = UserTokenKey let destinationKey = UserTokenKey() let legacyKey = LegacyTokenKey() func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { try await router.exists(legacyKey) } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { let legacyToken = try await router.get(legacyKey) try await router.set(legacyToken, for: destinationKey) try await router.remove(legacyKey) return MigrationResult(success: true, migratedCount: 1) } } // Associate with key extension UserTokenKey { var migration: AnyStorageMigration? { AnyStorageMigration(TokenMigration()) } } ``` ### Migration Types | Type | Use Case | |------|----------| | **SimpleLegacyMigration** | Direct data movement from legacy sources | | **ConditionalMigration** | Migrate based on app version or device state | | **TransformingMigration** | Convert data structures during migration | | **AggregatingMigration** | Combine multiple sources into one destination | ### 3. Use StorageRouter ```swift // Save let key = StorageKeys.UserTokenKey() try await StorageRouter.shared.set("token123", for: key) // Retrieve (triggers automatic migration if needed) let token = try await StorageRouter.shared.get(key) // Remove try await StorageRouter.shared.remove(key) ``` ### 4. Run Migration ```swift // Force migration let result = try await StorageRouter.shared.forceMigration(for: UserTokenKey()) ``` ``` ## Phase 6: Testing Implementation ### 6.1 New Test Files #### `Tests/LocalDataTests/MigrationProtocolTests.swift` ```swift @Suite(.serialized) struct MigrationProtocolTests { private let router: StorageRouter init() { // Fresh router for each test suite router = StorageRouter( keychain: MockKeychainHelper(), encryption: EncryptionHelper(keychain: MockKeychainHelper()), file: FileStorageHelper(configuration: FileStorageConfiguration(baseURL: testBaseURL)), defaults: UserDefaultsHelper(defaults: UserDefaults(suiteName: "MigrationProtocolTests.\(UUID().uuidString)")!) ) } @Test func simpleLegacyMigrationTest() async throws { // Test basic legacy migration behavior } @Test func conditionalMigrationTest() async throws { // Test conditional migration logic } @Test func transformingMigrationTest() async throws { // Test data transformation during migration } @Test func aggregatingMigrationTest() async throws { // Test multiple source aggregation } @Test func migrationErrorHandlingTest() async throws { // Test error scenarios and recovery } @Test func migrationResultValidationTest() async throws { // Test migration result structure } } ``` #### `Tests/LocalDataTests/MigrationIntegrationTests.swift` ```swift @Suite(.serialized) struct MigrationIntegrationTests { @Test func endToEndMigrationTest() async throws { // Test complete migration workflow } @Test func migrationRegistrationTest() async throws { // Ensure catalog registration and migration lookup work together } @Test func concurrentMigrationTest() async throws { // Test concurrent migration attempts } @Test func largeDataMigrationTest() async throws { // Test migration with large datasets } @Test func migrationFailureResultTest() async throws { // Test failure results and error reporting } } ``` ### 6.2 Update Existing Tests #### Modify `Tests/LocalDataTests/MigrationTests.swift` **Preserve Existing Tests:** - Keep all existing `MigrationTests` unchanged - Add new tests for protocol-based migrations - Ensure migration registration and catalog behavior stay consistent **New Test Methods:** ```swift @Test func protocolBasedMigrationTest() async throws { // Test new protocol system works } @Test func migrationRegistrationTest() async throws { // Test registration via catalog and direct API } ``` ### 6.3 Mock Implementations #### `Tests/LocalDataTests/Mocks/MockMigration.swift` ```swift /// Mock migration for testing struct MockMigration: StorageMigration { typealias DestinationKey = Destination let destinationKey: Destination let shouldSucceed: Bool let shouldMigrateResult: Bool let migrationDelay: TimeInterval func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { try await Task.sleep(for: .seconds(0.1)) return shouldMigrateResult } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { try await Task.sleep(for: .seconds(migrationDelay)) if shouldSucceed { return MigrationResult(success: true, migratedCount: 1) } else { return MigrationResult( success: false, errors: [.transformationFailed("Mock failure")] ) } } } /// Failing migration for error testing struct FailingMigration: StorageMigration { typealias DestinationKey = Destination let destinationKey: Destination let error: MigrationError func shouldMigrate(using router: StorageRouter, context: MigrationContext) async throws -> Bool { true } func migrate(using router: StorageRouter, context: MigrationContext) async throws -> MigrationResult { MigrationResult(success: false, errors: [error]) } } ``` ## Phase 7: Additional Utility Files ### 7.1 Device Information #### `Sources/LocalData/Utilities/DeviceInfo.swift` ```swift #if canImport(UIKit) import UIKit #endif #if os(watchOS) import WatchKit #endif /// Device information for migration context public struct DeviceInfo: Sendable { public let platform: Platform public let systemVersion: String public let model: String public let isSimulator: Bool public static let current = DeviceInfo() private init() { #if os(iOS) self.platform = .iOS self.systemVersion = UIDevice.current.systemVersion self.model = UIDevice.current.model #elseif os(watchOS) self.platform = .watchOS self.systemVersion = WKInterfaceDevice.current().systemVersion self.model = WKInterfaceDevice.current().model #else self.platform = .unknown self.systemVersion = ProcessInfo.processInfo.operatingSystemVersionString self.model = "Unknown" #endif self.isSimulator = ProcessInfo.processInfo.environment["SIMULATOR_DEVICE_NAME"] != nil } } public enum Platform: String, CaseIterable, Sendable { case iOS = "iOS" case watchOS = "watchOS" case unknown = "unknown" } ``` #### `Sources/LocalData/Utilities/SystemInfo.swift` ```swift /// System information for migration context public struct SystemInfo: Sendable { public let availableDiskSpace: UInt64 public let availableMemory: UInt64 public let isLowPowerModeEnabled: Bool public static let current = SystemInfo() private init() { // Get disk space let systemAttributes = try? FileManager.default.attributesOfFileSystem(forPath: NSHomeDirectory()) self.availableDiskSpace = (systemAttributes[.systemFreeSize] as? UInt64) ?? 0 // Get memory self.availableMemory = ProcessInfo.processInfo.physicalMemory // Get low power mode self.isLowPowerModeEnabled = ProcessInfo.processInfo.isLowPowerModeEnabled } } ``` ### 7.2 Migration Utilities #### `Sources/LocalData/Utilities/MigrationUtils.swift` ```swift /// Utilities for common migration operations public enum MigrationUtils { /// Check if data can be transformed from source to destination type public static func canTransform(from: T.Type, to: U.Type) -> Bool { // Check for common transformation patterns return T.self is U.Type || T.self == String.self || U.self == String.self } /// Get file size for migration considerations public static func estimatedSize(for data: Data) -> UInt64 { return UInt64(data.count) } /// Validate migration compatibility public static func validateCompatibility(source: StorageKeyDescriptor, destination: StorageKeyDescriptor) throws { // Check if domains are compatible // Check if types can be transformed // Validate security policies } } ``` ## Implementation Timeline | Phase | Duration | Description | |--------|-----------|-------------| | Phase 1: Core Protocol Infrastructure | 3-4 days | Create new protocols and models | | Phase 2: Concrete Migration Implementations | 2-3 days | Implement all migration types | | Phase 3: StorageRouter Integration | 2-3 days | Update router to use new protocol system | | Phase 4: Sample App Updates | 2-3 days | Demonstrate new capabilities in sample app | | Phase 5: Documentation Updates | 2 days | Comprehensive documentation overhaul | | Phase 6: Testing Implementation | 3-4 days | Comprehensive test coverage | | Phase 7: Additional Utilities | 1-2 days | Helper utilities and final polish | | **Total** | **15-21 days** | **Complete migration refactor** | ## Success Criteria 1. ✅ **Protocol-Based Migration System**: All migration types implemented and functional 2. ✅ **Migration Registration**: Catalogs and explicit registration compose correctly 3. ✅ **Sample App Demonstration**: All new migration types shown in working examples 4. ✅ **Documentation**: Comprehensive documentation and examples 5. ✅ **Test Coverage**: All new functionality thoroughly tested 6. ✅ **Performance**: Minimal impact on non-migration operations 7. ✅ **Error Handling**: Robust error reporting and recovery 8. ✅ **Migration History**: Complete tracking and prevention of re-migration ## Migration Checklist - [x] Create core protocol files - [x] Update StorageKey protocol with migration - [x] Create concrete migration implementations - [x] Update StorageRouter integration with simple and advanced migration handling - [x] Add WatchOS/sync defaults to StorageMigration - [x] Add migration registration methods - [x] Update sample app with new examples - [x] Create new storage keys for demo - [x] Implement comprehensive test suite including audits of existing tests, failure/rollback, watchOS scenarios - [x] Add mock implementations for testing - [x] Create utility files for device/system info - [x] Add migration history tracking - [x] Implement error handling and recovery - [x] Verify migration registration across multiple catalogs - [x] Performance testing with large datasets, including type erasure benchmarks - [x] Overhaul Migration.md documentation - [x] Update README.md with migration examples - [x] Final documentation review ## Notes ### Removal Notes - Remove legacy `migrationSources` usage from the codebase (no compatibility layer needed) - Update documentation to reflect the new migration surface only ### Performance Considerations - Conditional checks are fast before expensive operations - In-memory history tracking uses efficient lookup - Async operations for non-blocking migrations - Batch processing in AggregatingMigration for large data (e.g., streaming chunks) - Benchmarks in Phase 6 for type erasure overhead (e.g., compare AnyStorageMigration vs. potential 'any' alternatives); evaluate non-erased generics if issues found - Performance tests for >1MB migrations to ensure efficiency ### Error Handling - Comprehensive error types for all failure scenarios - Migration result provides detailed information - Rollback support for failed migrations - Recovery options for partial failures ### Testing Strategy - Preserve existing test patterns and coverage - Add new tests for all protocol types - Mock implementations for isolated testing - Integration tests for end-to-end workflows - Performance tests for large data migrations This refactor provides a powerful, flexible migration system while maintaining the excellent code quality and comprehensive test coverage of the existing LocalData package.