1
0
Fork 0
Commit graph

202 commits

Author SHA1 Message Date
Matthew Holt
9f2f69f2d3
Fix compilation
vips added an argument to Autorot().
2026-01-29 08:52:13 -07:00
Matthew Holt
c8cfe5001f
Fix more lint errors 2026-01-16 23:32:10 -07:00
Matthew Holt
d0b8e3f940
Include one more file for previous commit 2026-01-16 15:08:16 -07:00
Matthew Holt
ce601d5033
googlephotos: Ignore non-dir subfolders in Takeout albums
Also print data source in log messages when processing fails
2025-11-17 09:06:24 -07:00
Ralf Ebert
8218ab0a39
Don't use time_offset when searching items (#172)
Fix gallery photo prev/next for photos with time zone

When jumping between photos in the gallery with the prev/next button, the buttons were incorrectly enabled/disabled/wouldn't work even if enabled for me. This was caused by the photos having a time zone UTC+1 (time_offset = 3600). The js frontend uses formatted timestamps including the time zone (peekFromItem.timestamp in galleryFilterParams). These are converted to a unix timestamp via UTC().UnixMilli(), so the time zone is already taken care for and adding time_offset is not necessary. For the filtering in the gallery grid itself, a unix timestamp is generated client-side (something like &start=1762124400&end=1762210800) taking the system time zone into account, so this is correct without any extra logic as well.
2025-11-13 10:50:46 -07:00
Matthew Holt
59434bdde9
Why can't it just tell me all the lint errors, why leave off the last 2 2025-11-04 16:37:24 -07:00
Matthew Holt
2d9c8db9d7
ci: Fix lingering lint errors 2025-11-04 16:31:56 -07:00
Matthew Holt
962369382a
Hopefully appease new linter 2025-11-04 16:27:09 -07:00
Matthew Holt
7f3c90b71a
Continue WIP interactive mode 2025-10-24 00:00:43 -06:00
Matthew Holt
56e8037863
schema: Create index on embeddings(item_id)
Should fix a slow query in the hot path, for updated items
2025-10-23 20:53:04 -06:00
Matthew Holt
19acc6cc11
Create WAL checkpoint after jobs 2025-10-23 13:20:20 -06:00
Matthew Holt
03d126ed68
Use PRAGMA optimize instead of ANALYZE
This is supposedly a smarter way to do ANALYZE, as it only analyzes what sqlite thinks is needful. Should hopefully address some reports of too-frequent, long-running analyze queries.

There was one time I noticed that the pragma didn't improve query plans, until I ran analyze specifically which did improve it, but that was using the old DB connection model where I had a single pool of mixed readers/writers, so maybe it's possible that the new pooling style (separate r/w pools) also addresses that, I dunno.
2025-10-23 11:30:05 -06:00
Matthew Holt
e7650c784a
Some minor changes
- New config parameter "resume_jobs" which can disable auto-resuming jobs at timeline open. (closes #159)

- Renamed "a" to "app" in one method using "Rename symbol" (not "Change all occurrences"), which surprisingly updated the identifier in ALL methods. That must be new. Anyway, that's the huge diff.

- Minor fix to metadata merge that does a more proper nil check to avoid a panic.

- Changed some omitempty to omitzero
2025-10-22 15:13:32 -06:00
Matthew Holt
3bb6af98c1
Ensure meta map is not nil (fix #160) 2025-10-21 11:38:15 -06:00
Matthew Holt
e75213e841
Sanitize super-future timestamps (#145) 2025-10-13 11:03:50 -06:00
Matthew Holt
20f6c4a8f5 Sanitize zero-coordinates (fix #145) 2025-10-13 10:34:00 -06:00
Matthew Holt
0bf014cac0
kml: More rigorous filetype detection 2025-10-11 12:40:19 -06:00
Matthew Holt
1f1b60b8b1
Consider time.Local when processing update policies
Fixes unncessary item updates when repeating an import job
2025-10-10 21:27:53 -06:00
Matthew Holt
0d1d4311ae
Fix unnecessary errors in thumbnail generation 2025-10-10 14:25:52 -06:00
Matthew Holt
55b687a7aa
Make a hot path query more efficient 2025-10-10 07:23:49 -06:00
Matt Holt
41361913d1
ci: Attempt fix Linux workflows related to old libvips (#129)
Things would be so much easier if ubuntu just updated their packages

* ci: Attempt to fix Linux workflows

See if Copilot is worth its snuff

* Revert

* Try downgrading vipsgen instead

* Try again

* Try to install vips from source

* Sigh, ok try building from source

* Sigh

* sighhhh

* Sighhhhhhhhhhhhhhhhhhhh

* Try without cache for a moment

* Try caching again

* Try composite action

* Try again?

* Set shell on composite action steps...

* Update a couple other workflows

* Try to fix test job

* Some cleanup

* Add heif

* Oops

* Pointless comment but let's see if the cache worked

* Fix go builds

* Try installing pkg-config I guess?

* Try more pkg config paths?

* Tweak

* Are we there yet

* One more tweak

* Rename some things
2025-10-08 14:21:38 -06:00
Matthew Holt
94e963e779
Hopefully fix sqlite DSN for Windows 2025-10-08 11:17:15 -06:00
Matthew Holt
576d2c8b2a
Replace govips with vipsgen
See https://github.com/davidbyttow/govips?tab=readme-ov-file#the-recommended-path-forward-vipsgen
2025-10-08 10:36:55 -06:00
Matthew Holt
eaff29e1c3
Log data file download duration 2025-10-07 13:36:57 -06:00
Matthew Holt
6fa22bed8b
Support rendering of event class items 2025-10-07 09:53:31 -06:00
Matthew Holt
7dc9312237
Super minor fixes for obfuscation mode 2025-10-04 13:14:25 -06:00
Matthew Holt
9fc0c3e5c1
Work around Google Photos bug with missing ext on sidecar video files
Also fix motion picture transcoding for data files that don't have an extension, by looking up the media type of the image
2025-10-02 18:16:24 -06:00
Matthew Holt
0a70666bc5
Random initial in name for demo mode
Replace an initial in a name with another random letter, rather than a whole name string
2025-10-01 22:39:11 -06:00
Matthew Holt
1938dc2f1b
Replace frozen syscall package with x/sys 2025-10-01 12:29:43 -06:00
Matthew Holt
02868a90b7
Fix for determining if a folder is "empty"
When it contains multiple unintentional files
2025-09-30 21:40:08 -06:00
Matthew Holt
fb3d529228
Refactor thumbnail DB handle as well
Fix error when repo property doesn't exist
2025-09-30 14:04:53 -06:00
Matthew Holt
a4a4cfa370
Actually make exfat fix work
Oops
2025-09-30 13:04:07 -06:00
Matthew Holt
e87e5fcbaa
Fix spelling of function name 2025-09-30 12:49:15 -06:00
Matthew Holt
e9a7c03c53
Fix ExFAT crashes; refactor sql.DB handling
The crashes on ExFAT are caused by a bug in the MacOS ExFAT driver. It is unclear whether other OSes are affected too.

https://github.com/mattn/go-sqlite3/issues/1355

We now utilize sqlite's concurrency features by creating a write pool (size 1) and a read pool, and can eliminate our own RWMutex, which prevents reads at the same time as writes. Sqlite's WAL mode allows reads concurrent with writes, and our code is much cleaner.

Still need to do similar for the thumbnail DB.

Also could look into using prepared statements for more efficiency gains.
2025-09-30 12:31:41 -06:00
Matthew Holt
5994da8c75
Run ANALYZE less frequently; use write lock
Doubt this will fix the DB corruption errors, but, likely a good change anyway
2025-09-26 14:57:36 -06:00
Matthew Holt
039dfe5ba8
Fix and optimize entity processing; faster imports
Some certain rare edge cases were problematic, like when importing a contact list / vcard dataset after importing multiple messaging data sets, and there are entities with multiple phone numbers...

That, and a few other things are handled better. The loadEntities query has been cleaned up and corrected.

I got rid of autolink stuff with entity_attributes in the DB because it was not useful or really correct either. Added complexity causing bugs.

Imports are sometimes about 20-50% faster now.
2025-09-25 22:49:39 -06:00
Matthew Holt
64aaa2a13c
Support thumbnails for PDFs
Also a few very minor frontend fixes/tweaks
2025-09-25 22:44:57 -06:00
Matthew Holt
dab1adbf24
Force-update repo owner info when opening timeline
Bust the session storage cache in the browser

(Also load entity stored timestamp when loading entity)
2025-09-23 14:03:29 -06:00
Matthew Holt
1aed8ca2ca
Fix missing data files in some cases
The refactored processor had a bug where small, binary data files like images < 100 KB would be buffered entirely while peeking, and wouldn't end up being saved as a file. Fixed the logic around that and simplified a bit too.
2025-09-22 14:39:02 -06:00
Matthew Holt
93c3d5ab32
Fix edge case in some conversation queries
Need to open a paren if we close it
2025-09-20 14:39:49 -06:00
Matthew Holt
16a7d99fda
Actually fix map colors, kind of
The rendering seems inconsistent. If I refresh the page or load the results again, it fixes the color mismatches. I can't explain why they vary like this, other than potential mapbox bugs??
2025-09-20 10:08:42 -06:00
Matthew Holt
26f8b7da6f
Minor fixes; improve thumbnail job DB query speed
Also increase quality of transcoded videos on mac. 256k was horrendous. 512k is better but we might need 1024k.
2025-09-19 22:48:35 -06:00
Matthew Holt
3c40bbc182
Minor UI fixes 2025-09-19 09:37:00 -06:00
Matthew Holt
aba9f3a500
Fix obfuscated videos on Mac 2025-09-19 07:51:32 -06:00
Matthew Holt
6fef8ae50f
Fixes for obfuscated images and videos incl. thumbnails 2025-09-18 23:24:33 -06:00
Matthew Holt
aaaed9ab8d
One more fix 2025-09-18 21:20:34 -06:00
Matthew Holt
1568874a59
Minor fixes for obfuscation mode 2025-09-18 21:18:05 -06:00
Matthew Holt
11acad7d0e
Minor UI tweaks, increase video thumbnail quality 2025-09-18 17:50:53 -06:00
Matthew Holt
4bd1ae8856
Optionally generate thumbnails during import
This does away with the experimental generation of thumbhashes during import. It's easier to generate the thumbnails and thumbhashes at the same time.

Does add a DB lock to phase1, but at this point the DB isn't the bottleneck in that phase.
2025-09-18 17:37:53 -06:00
Matthew Holt
c8c1b65ce2
Try generating thumbhashes during import pipeline
Also show loading spinner for videos
2025-09-18 09:07:05 -06:00