Skip to content

Cotabby 2: cut the Open Source path to base models #497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FuJacob merged 2 commits into from
Jun 1, 2026
Merged

Cotabby 2: cut the Open Source path to base models #497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8f7f6e5
Cotabby 2: cut the Open Source path to base models, remove pipeline flag
FuJacob Jun 1, 2026
7970fbc
Remove dead code from the base-model migration
FuJacob Jun 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions Cotabby.xcodeproj/project.pbxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,8 @@
156E6AB3D24134EEC29FDB93 /* FocusSnapshotResolverSelectionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BA705EDFE1C41294F0E381F1 /* FocusSnapshotResolverSelectionTests.swift */; };
157A55EB796BEB7819B90D5D /* ClipboardRelevanceFilter.swift in Sources */ = {isa = PBXBuildFile; fileRef = D3A2AC525DC664DB540D4F19 /* ClipboardRelevanceFilter.swift */; };
15FA56CEF6FB5FF54C2FBA6F /* PermissionAndContextModelTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E7F42112F14026E6253BB865 /* PermissionAndContextModelTests.swift */; };
190C571B3CDFE117F4D15484 /* LlamaPromptRendererTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3009812A35A1CDEF16295AB7 /* LlamaPromptRendererTests.swift */; };
19CB55B62977376E9AE8D428 /* VisualContextStartCoalescer.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2F01FAC4F57EB08471521196 /* VisualContextStartCoalescer.swift */; };
1B3FFCB9A979F49BF86EAAD4 /* ScreenshotContextGeneratorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = B2BFD19A159680A495EE02FD /* ScreenshotContextGeneratorTests.swift */; };
1C4A2BAB2CCADF0A70B70AC6 /* LlamaPromptRenderer.swift in Sources */ = {isa = PBXBuildFile; fileRef = B5679E08C9A09065531C37B5 /* LlamaPromptRenderer.swift */; };
1D1C6FF0B8F50AC14A1000F4 /* SentenceBoundaryClassifierTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2D7360A6D4261989A66658ED /* SentenceBoundaryClassifierTests.swift */; };
1F8CC88AFFE67C08944CF506 /* WindowScreenshotService.swift in Sources */ = {isa = PBXBuildFile; fileRef = 77B0121E7BB173F8A2B0B108 /* WindowScreenshotService.swift */; };
2197B68F1E4D0C3497DAC061 /* LlamaSuggestionEngine.swift in Sources */ = {isa = PBXBuildFile; fileRef = BE04620C905041680116BE80 /* LlamaSuggestionEngine.swift */; };
Expand Down Expand Up @@ -221,6 +219,7 @@
E17CAA453B1F534D284F0D89 /* PermissionHostApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = B6ACCB12E4DB32D2F2BEA567 /* PermissionHostApp.swift */; };
E27E6377D36D4981301568DD /* LaunchAtLoginStateTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5807E8508D9355D0271A00C5 /* LaunchAtLoginStateTests.swift */; };
E313639E71AE1374D2B9A956 /* SuggestionWorkController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6B2D97BAA3618A7D0357AC44 /* SuggestionWorkController.swift */; };
E38801433B99E65BD7E45A0E /* LlamaPromptCacheHintTrackerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0CA88BB29BC8727878C99E95 /* LlamaPromptCacheHintTrackerTests.swift */; };
E4382BEA8A8551612E5966B9 /* BaseCompletionPromptRenderer.swift in Sources */ = {isa = PBXBuildFile; fileRef = 85EF79E6144D6C6AD062B569 /* BaseCompletionPromptRenderer.swift */; };
E51FA12B690428CA431328FC /* WritingPaneView.swift in Sources */ = {isa = PBXBuildFile; fileRef = D48B95B6665109B6C6A63B42 /* WritingPaneView.swift */; };
E6EE3C13FA31F261CD734C69 /* DownloadOutcomeClassifier.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3DE1975F3B5F4A70478DBF41 /* DownloadOutcomeClassifier.swift */; };
Expand Down Expand Up @@ -277,6 +276,7 @@
0A3D1125B962CBE0269EEDDB /* SuggestionInserter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionInserter.swift; sourceTree = "<group>"; };
0AC3BF78835C8F2C315932F1 /* EmojiCatalog.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiCatalog.swift; sourceTree = "<group>"; };
0C383AE85B971A9605787358 /* FocusModels.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusModels.swift; sourceTree = "<group>"; };
0CA88BB29BC8727878C99E95 /* LlamaPromptCacheHintTrackerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaPromptCacheHintTrackerTests.swift; sourceTree = "<group>"; };
0D80CC2CCAAFE3F23FB8C37A /* PromptContextSanitizerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PromptContextSanitizerTests.swift; sourceTree = "<group>"; };
0F5E263AB69029D5E13D5EE8 /* FocusDebugOverlayController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusDebugOverlayController.swift; sourceTree = "<group>"; };
110CB0B53016644EF7840301 /* HuggingFaceAPIClient.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HuggingFaceAPIClient.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -309,7 +309,6 @@
2D1F9CEBAB0F330F8E7B61D8 /* InputSuppressionController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InputSuppressionController.swift; sourceTree = "<group>"; };
2D7360A6D4261989A66658ED /* SentenceBoundaryClassifierTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SentenceBoundaryClassifierTests.swift; sourceTree = "<group>"; };
2F01FAC4F57EB08471521196 /* VisualContextStartCoalescer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = VisualContextStartCoalescer.swift; sourceTree = "<group>"; };
3009812A35A1CDEF16295AB7 /* LlamaPromptRendererTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaPromptRendererTests.swift; sourceTree = "<group>"; };
312C7306D916963F519CE0D9 /* EmojiTriggerStateMachine.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiTriggerStateMachine.swift; sourceTree = "<group>"; };
328847A0F494360033366791 /* TextDirectionDetector.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TextDirectionDetector.swift; sourceTree = "<group>"; };
3350EDE01ED5125520C79D53 /* SettingsCoordinator.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsCoordinator.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -423,7 +422,6 @@
B2F95847D76893C8A5B504B4 /* SuggestionOverlayStabilityGate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionOverlayStabilityGate.swift; sourceTree = "<group>"; };
B424E2AC97C99D335B0D5751 /* SuggestionTextNormalizer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SuggestionTextNormalizer.swift; sourceTree = "<group>"; };
B4B4A2E2DD6733658EC05BD8 /* DownloadFileRescuer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DownloadFileRescuer.swift; sourceTree = "<group>"; };
B5679E08C9A09065531C37B5 /* LlamaPromptRenderer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LlamaPromptRenderer.swift; sourceTree = "<group>"; };
B6ACCB12E4DB32D2F2BEA567 /* PermissionHostApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PermissionHostApp.swift; sourceTree = "<group>"; };
B6D42CD456B4B3C988B148A6 /* FocusTrackingModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FocusTrackingModel.swift; sourceTree = "<group>"; };
B7B185BA246A526CBA85E581 /* EmojiPickerPanelLayoutTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EmojiPickerPanelLayoutTests.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -753,7 +751,7 @@
43D627C4A55359EAF4676FF7 /* InsertionSafetyGateTests.swift */,
4793D4EA5D36D7E5CC216C27 /* LanguageSupportTests.swift */,
5807E8508D9355D0271A00C5 /* LaunchAtLoginStateTests.swift */,
3009812A35A1CDEF16295AB7 /* LlamaPromptRendererTests.swift */,
0CA88BB29BC8727878C99E95 /* LlamaPromptCacheHintTrackerTests.swift */,
52BAFA2F989C3C4F7FB892B5 /* MarkerSelectionSynthesizerTests.swift */,
1274F897631B1B3A835D157F /* MidWordContinuationPolicyTests.swift */,
FC83D14A7557BC0196E59007 /* MirrorOverlayLayoutTests.swift */,
Expand Down Expand Up @@ -907,7 +905,6 @@
41BBD5A4BA08CABE77860886 /* HardwareCapabilityProbe.swift */,
7D472F9F396672E57873303B /* InsertionSafetyGate.swift */,
EAAE6B395FAB604DF059280A /* KeyCodeLabels.swift */,
B5679E08C9A09065531C37B5 /* LlamaPromptRenderer.swift */,
8D610FCA3A97249DCCE7D0B8 /* LLMIOFileHandler.swift */,
A863F41C0C03D7B4AC5DC002 /* MarkerSelectionSynthesizer.swift */,
357C18383B047F24A531BDCD /* MidWordContinuationPolicy.swift */,
Expand Down Expand Up @@ -1134,7 +1131,6 @@
0A2DDD946654076675AC0FC6 /* LanguageCatalog.swift in Sources */,
51C069603DA16830868F1628 /* LanguageTagsEditor.swift in Sources */,
F0DEEE8A866ABB560E7A7E6A /* LaunchAtLoginService.swift in Sources */,
1C4A2BAB2CCADF0A70B70AC6 /* LlamaPromptRenderer.swift in Sources */,
66D9E37B12A9265D4733E72E /* LlamaRuntimeCore.swift in Sources */,
54BDF0D9C3DC7175555BD0F6 /* LlamaRuntimeManager.swift in Sources */,
4CAFD8F3444FEDC9ACAFF529 /* LlamaRuntimeModels.swift in Sources */,
Expand Down Expand Up @@ -1263,7 +1259,7 @@
83EC3543DC45B1601F119BF9 /* InsertionSafetyGateTests.swift in Sources */,
E912D4617AE1376061DF1F00 /* LanguageSupportTests.swift in Sources */,
E27E6377D36D4981301568DD /* LaunchAtLoginStateTests.swift in Sources */,
190C571B3CDFE117F4D15484 /* LlamaPromptRendererTests.swift in Sources */,
E38801433B99E65BD7E45A0E /* LlamaPromptCacheHintTrackerTests.swift in Sources */,
87806DE08881D11F2608A13D /* MarkerSelectionSynthesizerTests.swift in Sources */,
7C36DBA762E19C8C31676D44 /* MidWordContinuationPolicyTests.swift in Sources */,
14D77F0B8A195AC2FA8D24A9 /* MirrorOverlayLayoutTests.swift in Sources */,
Expand Down
102 changes: 47 additions & 55 deletions Cotabby/Models/LlamaRuntimeModels.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,70 +95,61 @@ struct DownloadableRuntimeModel: Equatable, Hashable, Sendable, Identifiable {
enum RuntimeModelCatalog {
static func displayName(for filename: String) -> String {
switch filename {
case "Qwen3-0.6B-Q4_K_M.gguf":
return "tabby-1-mini"
case "gemma-4-E2B-it-Q4_K_M.gguf":
return "tabby-1-base"
case "gemma-4-E4B-it-Q4_K_M.gguf":
return "tabby-1-pro"
case "SmolLM2-135M-Instruct-q8_0.gguf":
return "tabby-1-nano"
case "Qwen3.5-0.8B-Base.i1-Q6_K.gguf":
return "tabby-2-mini"
case "Qwen3.5-2B-Base.i1-Q4_K_M.gguf":
return "tabby-2-base"
case "Qwen3.5-4B-Base.i1-Q4_K_M.gguf":
return "tabby-2-pro"
case "gemma-4-E2B.i1-Q6_K.gguf":
return "tabby-2-gemma-mini"
case "gemma-4-E4B.i1-Q4_K_M.gguf":
return "tabby-2-gemma-pro"
default:
return filename
}
}

/// Canonical downloadable GGUF model list shown in Welcome and menu UI.
///
/// `expectedSizeBytes` and `sha256` were captured from HuggingFace's CDN
/// response headers (`x-linked-size` and `x-linked-etag` respectively).
/// To refresh after a model is updated upstream:
///
/// curl -sIL "<URL>" | grep -iE "^(x-linked-size|x-linked-etag):"
/// Builds a HuggingFace direct-download URL from a repo and file path.
private static func hfURL(_ repo: String, _ file: String) -> URL {
// Force-unwrap is safe: inputs are compile-time literals forming a valid URL.
URL(string: "https://huggingface.co/\(repo)/resolve/main/\(file)?download=true")!
}

/// Canonical downloadable base GGUF models for Cotabby 2's base-model continuation path.
/// Qwen3.5 / Gemma base checkpoints from mradermacher's i1 GGUF repos. `expectedSizeBytes` and
/// `sha256` stay nil pending CDN-header capture; the download manager skips size/hash
/// validation when they are nil. Old instruct GGUFs are intentionally no longer listed.
static let downloadableModels: [DownloadableRuntimeModel] = [
DownloadableRuntimeModel(
filename: "SmolLM2-135M-Instruct-q8_0.gguf",
displayName: displayName(for: "SmolLM2-135M-Instruct-q8_0.gguf"),
downloadURL: URL(
string:
"https://huggingface.co/Mungert/SmolLM2-135M-Instruct-GGUF/resolve/main/SmolLM2-135M-Instruct-q8_0.gguf?download=true"
)!,
approximateSizeInGigabytes: 0.1,
expectedSizeBytes: 144_811_552,
sha256: "bc64cce8e1c11e4ed870633b557e04af718249c817c4cf8a6784116144ec3e28"
filename: "Qwen3.5-0.8B-Base.i1-Q6_K.gguf",
displayName: displayName(for: "Qwen3.5-0.8B-Base.i1-Q6_K.gguf"),
downloadURL: hfURL("mradermacher/Qwen3.5-0.8B-Base-i1-GGUF", "Qwen3.5-0.8B-Base.i1-Q6_K.gguf"),
approximateSizeInGigabytes: 0.8
),
DownloadableRuntimeModel(
filename: "Qwen3.5-2B-Base.i1-Q4_K_M.gguf",
displayName: displayName(for: "Qwen3.5-2B-Base.i1-Q4_K_M.gguf"),
downloadURL: hfURL("mradermacher/Qwen3.5-2B-Base-i1-GGUF", "Qwen3.5-2B-Base.i1-Q4_K_M.gguf"),
approximateSizeInGigabytes: 1.4
),
DownloadableRuntimeModel(
filename: "Qwen3-0.6B-Q4_K_M.gguf",
displayName: displayName(for: "Qwen3-0.6B-Q4_K_M.gguf"),
downloadURL: URL(
string:
"https://huggingface.co/unsloth/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q4_K_M.gguf?download=true"
)!,
approximateSizeInGigabytes: 0.4,
expectedSizeBytes: 396_705_472,
sha256: "ac2d97712095a558e31573f62f466a3f9d93990898b0ec79d7c974c1780d524a"
filename: "Qwen3.5-4B-Base.i1-Q4_K_M.gguf",
displayName: displayName(for: "Qwen3.5-4B-Base.i1-Q4_K_M.gguf"),
downloadURL: hfURL("mradermacher/Qwen3.5-4B-Base-i1-GGUF", "Qwen3.5-4B-Base.i1-Q4_K_M.gguf"),
approximateSizeInGigabytes: 2.6
),
DownloadableRuntimeModel(
filename: "gemma-4-E2B-it-Q4_K_M.gguf",
displayName: displayName(for: "gemma-4-E2B-it-Q4_K_M.gguf"),
downloadURL: URL(
string:
"https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF/resolve/main/gemma-4-E2B-it-Q4_K_M.gguf?download=true"
)!,
approximateSizeInGigabytes: 3.1,
expectedSizeBytes: 3_106_736_256,
sha256: "9378bc471710229ef165709b62e34bfb62231420ddaf6d729e727305b5b8672d"
filename: "gemma-4-E2B.i1-Q6_K.gguf",
displayName: displayName(for: "gemma-4-E2B.i1-Q6_K.gguf"),
downloadURL: hfURL("mradermacher/gemma-4-E2B-i1-GGUF", "gemma-4-E2B.i1-Q6_K.gguf"),
approximateSizeInGigabytes: 4.5
),
DownloadableRuntimeModel(
filename: "gemma-4-E4B-it-Q4_K_M.gguf",
displayName: displayName(for: "gemma-4-E4B-it-Q4_K_M.gguf"),
downloadURL: URL(
string:
"https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF/resolve/main/gemma-4-E4B-it-Q4_K_M.gguf?download=true"
)!,
approximateSizeInGigabytes: 5.0,
expectedSizeBytes: 4_977_169_568,
sha256: "519b9793ed6ce0ff530f1b7c96e848e08e49e7af4d57bb97f76215963a54146d"
filename: "gemma-4-E4B.i1-Q4_K_M.gguf",
displayName: displayName(for: "gemma-4-E4B.i1-Q4_K_M.gguf"),
downloadURL: hfURL("mradermacher/gemma-4-E4B-i1-GGUF", "gemma-4-E4B.i1-Q4_K_M.gguf"),
approximateSizeInGigabytes: 5.0
)
]
}
Expand All @@ -176,10 +167,11 @@ struct LlamaRuntimeConfiguration: Equatable, Sendable {
static let `default` = LlamaRuntimeConfiguration(
runtimeDirectoryPath: nil,
preferredModelNames: [
"gemma-4-E4B-it-Q4_K_M.gguf",
"gemma-4-E2B-it-Q4_K_M.gguf",
"Qwen3-0.6B-Q4_K_M.gguf",
"SmolLM2-135M-Instruct-q8_0.gguf"
"Qwen3.5-2B-Base.i1-Q4_K_M.gguf",
"Qwen3.5-0.8B-Base.i1-Q6_K.gguf",
"Qwen3.5-4B-Base.i1-Q4_K_M.gguf",
"gemma-4-E2B.i1-Q6_K.gguf",
"gemma-4-E4B.i1-Q4_K_M.gguf"
],
contextWindowTokens: 2048,
batchSize: 512,
Expand Down
6 changes: 3 additions & 3 deletions Cotabby/Models/OnboardingTemplate.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,11 @@ enum OnboardingTemplate: String, CaseIterable, Identifiable, Equatable, Sendable
var openSourceModelFilename: String {
switch self {
case .quick:
return "SmolLM2-135M-Instruct-q8_0.gguf"
return "Qwen3.5-0.8B-Base.i1-Q6_K.gguf"
case .everyday:
return "gemma-4-E2B-it-Q4_K_M.gguf"
return "Qwen3.5-2B-Base.i1-Q4_K_M.gguf"
case .powerful:
return "gemma-4-E4B-it-Q4_K_M.gguf"
return "Qwen3.5-4B-Base.i1-Q4_K_M.gguf"
}
}
}
Expand Down
6 changes: 1 addition & 5 deletions Cotabby/Models/SuggestionEngineModels.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ struct SuggestionSettingsSnapshot: Equatable, Sendable {
let selectedEngine: SuggestionEngineKind
let selectedWordCountPreset: SuggestionWordCountPreset
let isClipboardContextEnabled: Bool
/// User-authored profile data for Cotabby's single instruction-rendered completion prompt.
/// User-authored profile data for Cotabby's base-model completion prompt.
/// This travels in the snapshot so generation uses the same value the Settings UI shows.
let userName: String
/// User-authored style rules, carried in the snapshot so generation uses the same value the
Expand All @@ -86,10 +86,6 @@ struct SuggestionSettingsSnapshot: Equatable, Sendable {
/// When true, the screenshot/OCR visual-context pipeline is skipped entirely for lower-latency
/// suggestions. Defaults to false. Only affects visual context — predictions still run.
let isFastModeEnabled: Bool
/// Experimental: when true and the Open Source engine is selected, the local path uses the
/// base-model continuation prompt (no instruction preamble, prefix last) instead of the
/// instruction-rendered prompt. Default false, so existing installs are byte-for-byte unchanged.
let useBaseCompletionPipeline: Bool
/// User preference for how suggestions are presented (inline ghost text vs popup card vs auto
/// based on caret geometry quality). Travels in the snapshot so consumers can react to changes
/// without subscribing to the settings model directly.
Expand Down
Loading