Skip to content

Speech for PDF documents #1131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 23 commits into
base: master
Choose a base branch
from
Draft

Speech for PDF documents #1131

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b75c356
WIP speech controllers
michalrentka Mar 11, 2025
a2c81d7
Basic speech manager implemented
michalrentka Mar 12, 2025
2cb4e5c
WIP: forward/backward mechanic
michalrentka Mar 19, 2025
e493487
WIP
michalrentka Mar 27, 2025
0f20666
WIP
michalrentka Apr 8, 2025
11b7b3e
Improved paragraph sorting
michalrentka May 6, 2025
8cd8a5a
WIP
michalrentka May 15, 2025
268600b
Added the option to load single page to pdf-worker
michalrentka May 20, 2025
83a4d0a
WIP
michalrentka May 22, 2025
54b8fc1
Replaced text extraction with pdf worker
michalrentka May 22, 2025
1591220
Fixed pdf worker to fetch pdf text
michalrentka May 23, 2025
a615d46
Fixed going backwards
michalrentka May 28, 2025
ec3eebd
wip
michalrentka Jun 4, 2025
c4e1725
Bug fixes
michalrentka Jun 6, 2025
6acfd88
WIP
michalrentka Jun 10, 2025
85de335
Final UI adjustments, bug fixes
michalrentka Jun 13, 2025
703b8bc
UI adjustments and fixes
michalrentka Jun 17, 2025
308abf5
Accessibility improvements
michalrentka Jun 18, 2025
9a204ea
Fix realm dependency
michalrentka Jun 18, 2025
9bc9206
Removed old code
michalrentka Jun 18, 2025
4fd1d71
wip
michalrentka Jun 26, 2025
c8e2c4a
UI improvements
michalrentka Jun 26, 2025
fb12d6a
iPad UI finished
michalrentka Jul 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions Zotero.xcodeproj/project.pbxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -802,6 +802,8 @@
B372CEE7248673DF00B423AE /* GroupRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = B372CEE22486512500B423AE /* GroupRequest.swift */; };
B372CEE92486748500B423AE /* MarkGroupForResyncSyncAction.swift in Sources */ = {isa = PBXBuildFile; fileRef = B372CEE82486748500B423AE /* MarkGroupForResyncSyncAction.swift */; };
B372CEEA248675A200B423AE /* MarkGroupForResyncSyncAction.swift in Sources */ = {isa = PBXBuildFile; fileRef = B372CEE82486748500B423AE /* MarkGroupForResyncSyncAction.swift */; };
B37352AF2E140DF10078BB00 /* AccessibilityViewHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = B37352AE2E140DE70078BB00 /* AccessibilityViewHandler.swift */; };
B37352B12E141F9F0078BB00 /* AccessibilityReaderOverlayView.swift in Sources */ = {isa = PBXBuildFile; fileRef = B37352B02E141F990078BB00 /* AccessibilityReaderOverlayView.swift */; };
B373877328FEA0C7004E5031 /* PDFSidebarViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = B373877228FEA0C7004E5031 /* PDFSidebarViewController.swift */; };
B373B1B82BBD74F400DCF893 /* DeleteFailedItemsDbRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = B373B1B62BBD748600DCF893 /* DeleteFailedItemsDbRequest.swift */; };
B373C98F2B1F5431007FD56C /* PDFThumbnailsLayout.swift in Sources */ = {isa = PBXBuildFile; fileRef = B373C98E2B1F5431007FD56C /* PDFThumbnailsLayout.swift */; };
Expand Down Expand Up @@ -857,6 +859,7 @@
B3830CEA255451DC00910FE0 /* TagPickerViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = B3830CE8255451DC00910FE0 /* TagPickerViewController.swift */; };
B3830CF625545EE400910FE0 /* TagPickerCell.swift in Sources */ = {isa = PBXBuildFile; fileRef = B3830CF425545EE400910FE0 /* TagPickerCell.swift */; };
B3830CF725545EE400910FE0 /* TagPickerCell.xib in Resources */ = {isa = PBXBuildFile; fileRef = B3830CF525545EE400910FE0 /* TagPickerCell.xib */; };
B384B5C22E1539D000D023C6 /* AccessibilitySpeechControlsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = B384B5C12E1539C900D023C6 /* AccessibilitySpeechControlsView.swift */; };
B385B70E25C03E7E0073CA6F /* PDFExportState.swift in Sources */ = {isa = PBXBuildFile; fileRef = B385B70D25C03E7E0073CA6F /* PDFExportState.swift */; };
B386328626C5499900183062 /* TranslatorsAndStylesController.swift in Sources */ = {isa = PBXBuildFile; fileRef = B36A988C2428E059005D5790 /* TranslatorsAndStylesController.swift */; };
B3863FC82AD819AB005082F0 /* EndItemCreationDbRequest.swift in Sources */ = {isa = PBXBuildFile; fileRef = B3863FC72AD819AB005082F0 /* EndItemCreationDbRequest.swift */; };
Expand Down Expand Up @@ -887,6 +890,7 @@
B395EFE725494B2300CEBD9F /* CreatorEditState.swift in Sources */ = {isa = PBXBuildFile; fileRef = B395EFE625494B2300CEBD9F /* CreatorEditState.swift */; };
B395EFEB25494B3200CEBD9F /* CreatorEditActionHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = B395EFEA25494B3200CEBD9F /* CreatorEditActionHandler.swift */; };
B39649182869B0D0000BCB6C /* ISBNParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39649172869B0D0000BCB6C /* ISBNParser.swift */; };
B3969A402DE9AAEF00425FF3 /* AccessibilityPopupViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = B3969A3F2DE9AAEF00425FF3 /* AccessibilityPopupViewController.swift */; };
B3972689247D403200A8B469 /* UrlDetector.swift in Sources */ = {isa = PBXBuildFile; fileRef = B3972688247D403200A8B469 /* UrlDetector.swift */; };
B3981427257A5A16002C755C /* AnnotationView.swift in Sources */ = {isa = PBXBuildFile; fileRef = B3981426257A5A16002C755C /* AnnotationView.swift */; };
B398142E257A649D002C755C /* TextContentEditCell.swift in Sources */ = {isa = PBXBuildFile; fileRef = B398142C257A649D002C755C /* TextContentEditCell.swift */; };
Expand All @@ -906,6 +910,7 @@
B39D42DF29C0CE7D0035CDA9 /* TagsFlowLayout.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39D42DE29C0CE7D0035CDA9 /* TagsFlowLayout.swift */; };
B39DBFF32D7EF2C600454486 /* FeatureGates.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39DBFF22D7EF2C600454486 /* FeatureGates.swift */; };
B39DBFF42D7EF2C600454486 /* FeatureGates.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39DBFF22D7EF2C600454486 /* FeatureGates.swift */; };
B39DBFF62D8036A000454486 /* SpeechManager.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39DBFF52D80369C00454486 /* SpeechManager.swift */; };
B39E0132283276830091CE4A /* WebViewHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39E0131283276830091CE4A /* WebViewHandler.swift */; };
B39E013328327CAC0091CE4A /* WebViewHandler.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39E0131283276830091CE4A /* WebViewHandler.swift */; };
B39EE32B223A5E3500302E29 /* TestAppDelegate.swift in Sources */ = {isa = PBXBuildFile; fileRef = B39EE32A223A5E3500302E29 /* TestAppDelegate.swift */; };
Expand Down Expand Up @@ -1876,6 +1881,8 @@
B372CEE22486512500B423AE /* GroupRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = GroupRequest.swift; sourceTree = "<group>"; };
B372CEE4248670AF00B423AE /* FetchAndStoreGroupSyncAction.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FetchAndStoreGroupSyncAction.swift; sourceTree = "<group>"; };
B372CEE82486748500B423AE /* MarkGroupForResyncSyncAction.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MarkGroupForResyncSyncAction.swift; sourceTree = "<group>"; };
B37352AE2E140DE70078BB00 /* AccessibilityViewHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AccessibilityViewHandler.swift; sourceTree = "<group>"; };
B37352B02E141F990078BB00 /* AccessibilityReaderOverlayView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AccessibilityReaderOverlayView.swift; sourceTree = "<group>"; };
B373877228FEA0C7004E5031 /* PDFSidebarViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PDFSidebarViewController.swift; sourceTree = "<group>"; };
B373B1B62BBD748600DCF893 /* DeleteFailedItemsDbRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DeleteFailedItemsDbRequest.swift; sourceTree = "<group>"; };
B373C98E2B1F5431007FD56C /* PDFThumbnailsLayout.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PDFThumbnailsLayout.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -1923,6 +1930,7 @@
B3830CE8255451DC00910FE0 /* TagPickerViewController.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = TagPickerViewController.swift; sourceTree = "<group>"; };
B3830CF425545EE400910FE0 /* TagPickerCell.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TagPickerCell.swift; sourceTree = "<group>"; };
B3830CF525545EE400910FE0 /* TagPickerCell.xib */ = {isa = PBXFileReference; lastKnownFileType = file.xib; path = TagPickerCell.xib; sourceTree = "<group>"; };
B384B5C12E1539C900D023C6 /* AccessibilitySpeechControlsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AccessibilitySpeechControlsView.swift; sourceTree = "<group>"; };
B385B70D25C03E7E0073CA6F /* PDFExportState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PDFExportState.swift; sourceTree = "<group>"; };
B3863FC72AD819AB005082F0 /* EndItemCreationDbRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EndItemCreationDbRequest.swift; sourceTree = "<group>"; };
B3863FC92AD830DE005082F0 /* DeleteCreatorItemDetailDbRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DeleteCreatorItemDetailDbRequest.swift; sourceTree = "<group>"; };
Expand All @@ -1946,6 +1954,7 @@
B395EFE625494B2300CEBD9F /* CreatorEditState.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CreatorEditState.swift; sourceTree = "<group>"; };
B395EFEA25494B3200CEBD9F /* CreatorEditActionHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CreatorEditActionHandler.swift; sourceTree = "<group>"; };
B39649172869B0D0000BCB6C /* ISBNParser.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ISBNParser.swift; sourceTree = "<group>"; };
B3969A3F2DE9AAEF00425FF3 /* AccessibilityPopupViewController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AccessibilityPopupViewController.swift; sourceTree = "<group>"; };
B3972688247D403200A8B469 /* UrlDetector.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = UrlDetector.swift; sourceTree = "<group>"; };
B3981426257A5A16002C755C /* AnnotationView.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = AnnotationView.swift; sourceTree = "<group>"; };
B398142C257A649D002C755C /* TextContentEditCell.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TextContentEditCell.swift; sourceTree = "<group>"; };
Expand All @@ -1964,6 +1973,7 @@
B39D42D829C0B82F0035CDA9 /* TagFilterCell.xib */ = {isa = PBXFileReference; lastKnownFileType = file.xib; path = TagFilterCell.xib; sourceTree = "<group>"; };
B39D42DE29C0CE7D0035CDA9 /* TagsFlowLayout.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TagsFlowLayout.swift; sourceTree = "<group>"; };
B39DBFF22D7EF2C600454486 /* FeatureGates.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FeatureGates.swift; sourceTree = "<group>"; };
B39DBFF52D80369C00454486 /* SpeechManager.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SpeechManager.swift; sourceTree = "<group>"; };
B39E0131283276830091CE4A /* WebViewHandler.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WebViewHandler.swift; sourceTree = "<group>"; };
B39EE32A223A5E3500302E29 /* TestAppDelegate.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TestAppDelegate.swift; sourceTree = "<group>"; };
B3A05E94261492150077CCCC /* UpdateCollectionLastUsedDbRequest.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = UpdateCollectionLastUsedDbRequest.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -2439,6 +2449,7 @@
B305646F23FC051E003304F2 /* SecureStorage.swift */,
B305646E23FC051E003304F2 /* SessionController.swift */,
61ABA7502A6137D1002A4219 /* ShareableImage.swift */,
B39DBFF52D80369C00454486 /* SpeechManager.swift */,
B3593F172668E6A700FA4BB2 /* StyleParserDelegate.swift */,
B355B12B2850B6C400BAE2C5 /* TableViewDiffableDataSource.swift */,
B36BEC7224485FC700A60552 /* TagColorGenerator.swift */,
Expand Down Expand Up @@ -3976,6 +3987,10 @@
B3DCDF00240912060039ED0D /* Views */ = {
isa = PBXGroup;
children = (
B384B5C12E1539C900D023C6 /* AccessibilitySpeechControlsView.swift */,
B3969A3F2DE9AAEF00425FF3 /* AccessibilityPopupViewController.swift */,
B37352B02E141F990078BB00 /* AccessibilityReaderOverlayView.swift */,
B37352AE2E140DE70078BB00 /* AccessibilityViewHandler.swift */,
B3DCDF03240912060039ED0D /* ActivityIndicatorView.swift */,
B347F2B924DC4D5D000D5FF6 /* CapHeightLabel.swift */,
B3D58D6A25EE437700D8FA31 /* CircularProgressView.swift */,
Expand Down Expand Up @@ -5027,6 +5042,7 @@
B305661023FC051E003304F2 /* SchemaRequest.swift in Sources */,
B305660223FC051E003304F2 /* ApiClient.swift in Sources */,
B312BB4125519DD30023300A /* ItemDetailLayout.swift in Sources */,
B37352B12E141F9F0078BB00 /* AccessibilityReaderOverlayView.swift in Sources */,
B331F9B5265400D70099F6A6 /* ReadStylesDbRequest.swift in Sources */,
B39D33712400150B00EF2ACB /* CreateAttachmentsDbRequest.swift in Sources */,
B3A081E125D2D8050088EC24 /* CollectionEditHostingViewController.swift in Sources */,
Expand Down Expand Up @@ -5109,6 +5125,7 @@
61EB05B42C9D8EA200152482 /* CreateReaderAnnotationsDbRequest.swift in Sources */,
B3229FD128C0A07500DAF3B7 /* EditAnnotationRectsDbRequest.swift in Sources */,
B3DF9AD22747AAD2007933CB /* ApiRequest.swift in Sources */,
B384B5C22E1539D000D023C6 /* AccessibilitySpeechControlsView.swift in Sources */,
615FD8772C8B475A00808C1D /* ReaderCoordinator.swift in Sources */,
B3DCDF0E240912500039ED0D /* SinglePickerActionHandler.swift in Sources */,
B3593F162668E29C00FA4BB2 /* Style.swift in Sources */,
Expand Down Expand Up @@ -5373,6 +5390,7 @@
B3DAF84D2AC5AA5C00F9AB6A /* PDFDocumentAnnotation.swift in Sources */,
B361C2B429DECB220035BF7E /* ResetSettingsVersionDbRequest.swift in Sources */,
B30565B623FC051E003304F2 /* StoreItemsDbResponseRequest.swift in Sources */,
B37352AF2E140DF10078BB00 /* AccessibilityViewHandler.swift in Sources */,
B395EFEB25494B3200CEBD9F /* CreatorEditActionHandler.swift in Sources */,
B30565E623FC051E003304F2 /* SchemaController.swift in Sources */,
B339459126DE6C2A00E59A02 /* AttachmentFileDeletedNotification.swift in Sources */,
Expand Down Expand Up @@ -5460,6 +5478,7 @@
B3DEAAA728AD01CA00F72D90 /* URLSession+Extensions.swift in Sources */,
B36FD9B32A7929C8002D77E8 /* FontSizeCell.swift in Sources */,
B357A28C285B73BD00E73CA1 /* ScannerState.swift in Sources */,
B3969A402DE9AAEF00425FF3 /* AccessibilityPopupViewController.swift in Sources */,
B3E8FE5A2714325200F51458 /* SavingSettingsActionHandler.swift in Sources */,
B30566B623FC051F003304F2 /* DeletionsResponse.swift in Sources */,
B30566C423FC051F003304F2 /* DeleteBatch.swift in Sources */,
Expand All @@ -5480,6 +5499,7 @@
B34341B4260A496200093E63 /* ReadCollectionAndLibraryDbRequest.swift in Sources */,
B30566BA23FC051F003304F2 /* Defaults.swift in Sources */,
B3325F6F27B41D69007C7137 /* CollectionTree.swift in Sources */,
B39DBFF62D8036A000454486 /* SpeechManager.swift in Sources */,
B30565E423FC051E003304F2 /* RealmDbStorage.swift in Sources */,
B3E8FE742714344D00F51458 /* SettingsListView.swift in Sources */,
B3343079282A563B00FB41BC /* NSUserActivity+Activities.swift in Sources */,
Expand Down
11 changes: 10 additions & 1 deletion Zotero/Assets/en.lproj/Localizable.strings
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -576,9 +576,18 @@
"accessibility.pdf.annotation_hint" = "Double-tap to select and edit";
"accessibility.pdf.tags_hint" = "Double-tap to edit tags";
"accessibility.pdf.selected" = "Selected";
"accessibility.pdf.open_reader" = "Open reading mode";
"accessibility.pdf.search_pdf" = "Search PDF";
"accessibility.pdf.redo" = "Redo";
"accessibility.pdf.undo" = "Undo";
"accessibility.pdf.toggle_annotation_toolbar" = "Toggle annotation toolbar";
"accessibility.pdf.show_more_tools" = "Show more";
"accessibility.speech.play" = "Start speaking";
"accessibility.speech.pause" = "Pause speaking";
"accessibility.speech.forward" = "Skip forward";
"accessibility.speech.backward" = "Skip backward";
"accessibility.speech.title" = "Read Aloud";
"accessibility.show_reader" = "Reading Mode";
"accessibility.show_reader_accessibility_label" = "Switch to reading mode";
"accessibility.show_speech" = "Read Aloud";
"accessibility.show_speech_accessibility_label" = "Start reading aloud";
"accessibility.open_document_accessibility" = "Open accessibility options";
8 changes: 4 additions & 4 deletions Zotero/Controllers/PDFWorkerController.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ import RxSwift
final class PDFWorkerController {
// MARK: Types
struct PDFWork: Hashable {
enum Kind {
enum Kind: Hashable {
case recognizer
case fullText
case fullText(pages: [Int])
}

let file: FileData
Expand Down Expand Up @@ -113,8 +113,8 @@ final class PDFWorkerController {
case .recognizer:
pdfWorkerWebViewHandler.recognize(file: work.file)

case .fullText:
pdfWorkerWebViewHandler.getFullText(file: work.file)
case .fullText(let pages):
pdfWorkerWebViewHandler.getFullText(file: work.file, pages: pages)
}

func setupObserver(for pdfWorkerWebViewHandler: PDFWorkerWebViewHandler) {
Expand Down
Loading
Loading