11. Performance, Memory, dan Battery

Foundation Models

First-token latency: ~200-500ms.
Streaming throughput: ~30-50 token/sec.
Memory: model di-share sistem, app overhead ~50-100MB.
Battery: lebih hemat dari cloud (no radio), tapi sustained inference panas → throttling.

Core ML

Latency: 1ms-1s tergantung model.
Memory: model + tensor; bisa load on-demand via MLModelConfiguration.computeUnits.
Battery: Neural Engine paling hemat. Pilih computeUnits: .all (default) atau .cpuAndNeuralEngine.

Cloud LLM

Latency: 500ms-3s + variability network.
Memory: minimal (cuma stream parsing).
Battery: radio + payload — relatif boros di seluler.
Cost: $0.001-0.10/req. Multiply per DAU × interaksi per user.

Optimasi Praktis

swift
// performance: Cache hasil AI yang mahal
actor AIResultCache {
    private var cache: [String: String] = [:]
    private let maxEntries = 100

    func result(for prompt: String) -> String? { cache[prompt] }

    func store(_ result: String, for prompt: String) {
        if cache.count >= maxEntries {
            cache.removeValue(forKey: cache.keys.randomElement()!)
        }
        cache[prompt] = result
    }
}

swift
// performance: Debounce request AI yang trigger dari typing
@MainActor
final class AISuggestionVM {
    var input: String = ""
    var suggestions: [String] = []
    private var task: Task<Void, Never>?

    func update(_ newInput: String) {
        input = newInput
        task?.cancel()
        task = Task { [weak self] in
            try? await Task.sleep(for: .milliseconds(500))  // wait for typing pause
            guard !Task.isCancelled, let self else { return }
            self.suggestions = await generateSuggestions(self.input)
        }
    }
}

swift
// performance: Limit context length untuk LLM
func compactHistory(_ messages: [Message], maxTokens: Int = 4000) -> [Message] {
    // Keep system message + recent messages within budget
    let estimated = messages.reduce(0) { $0 + estimateTokens($1.content) }
    if estimated <= maxTokens { return messages }
    // Drop dari tengah, keep system + recent
    return Array(messages.prefix(1)) + Array(messages.suffix(messages.count - 3))
}