1
0
Fork 0
Commit graph

202 commits

Author SHA1 Message Date
Matthew Holt
6a10d23a7c
Fix: timestamps, coordinate precision, map loading
- Timestamp year cannot be > 9999 (JSON serialization panics)
- Lat/lon now considered equivalent after a certain decimal point, since not all sources have high precision (we choose 5 decimal points for now, or about 1.1 meters)
- Map style must be loaded before source is added, apparently (got this error once)
2025-05-14 14:41:08 -06:00
Matthew Holt
360e131fff
Recover panics during jobs/imports, and support base64 pics from vCard 2025-05-14 08:29:37 -06:00
Matthew Holt
e9fc2eb022
Couple of bug fixes 2025-05-12 20:05:19 -06:00
Matthew Holt
c8bbe58358
The default repo folder name should probably be capitalized 2025-05-12 19:40:48 -06:00
Matthew Holt
f3f4f1bc22
Use same name obfuscation for related entities as standalone entities 2025-05-12 15:56:14 -06:00
Matthew Holt
874be1a9ca
Add UI for unique constraints and item update preferences 2025-05-12 12:34:48 -06:00
Matthew Holt
ae3a5d02b0
Field update preferences allow more control over item updates 2025-05-09 10:04:03 -06:00
Matthew Holt
ffc8ad6f51
applephotos: Preserve a lot more metadata about people in photos
Also infer owner entity from DB if necessary, very cool!

Also fix a couple minor bugs
2025-05-05 14:48:54 -06:00
Matthew Holt
a62f4aa05a
applephotos: Initial commit of Apple Photos data source
Still a WIP, but mostly there!
2025-05-05 12:07:13 -06:00
Matthew Holt
ba4635cf7e
Fix data file handling
It wasn't updated properly with the big pipeline refactor
2025-05-04 13:28:20 -06:00
Matthew Holt
15c55f0a8f
Improve pause/unpause behavior 2025-05-02 08:55:27 -06:00
Matthew Holt
72c8ede971
More improvements/fixes to thumbnail jobs 2025-05-01 22:18:50 -06:00
Matthew Holt
3d2222fce2
Fix thumbnail job size count and paging; other minor fixes
Including one fix for a panic introduced by obfuscated logging during processing
2025-05-01 11:15:13 -06:00
Matthew Holt
914f24f6a6
Move calls to pythonServerReady 2025-04-28 21:08:16 -06:00
Matthew Holt
98069ee66b
Health check for python server
This ensures that searches and embedding jobs don't run before it has fully loaded, since loading models can take a while
2025-04-28 17:33:40 -06:00
Matthew Holt
25712e7c61
Fix thumbnail job size counts 2025-04-28 10:26:59 -06:00
Matthew Holt
38c89f2b0a
Only prepare finished graph log if needful; fix obfuscation bug 2025-04-27 07:49:34 -06:00
Matthew Holt
e760b594e0
Sigh, fix linter 2025-04-25 22:07:19 -06:00
Matthew Holt
e0d5bdb67c
Obfuscate entity names a little more intelligently 2025-04-25 22:06:09 -06:00
Matthew Holt
f0697d2d6b
Refactor embedding jobs; enhance tooltips; upgrade gofakeit to v7
The gofakeit upgrade uses the new math/rand/v2 package, which uses uint64 more than int64, so we had to change a bunch of row IDs from int64 to uint64.
2025-04-24 16:33:41 -06:00
Matthew Holt
516eef9b4d
Fix lint warning 2025-04-22 08:40:03 -06:00
Matthew Holt
06656c8ab7
ci: Ignore obfuscation file
Many magic numbers, doesn't matter, don't care.
2025-04-22 07:52:10 -06:00
Matthew Holt
af689078ee
Nicer tooltip in message display
(And rename file to make more sense.)
2025-04-21 22:00:14 -06:00
Matthew Holt
167585bae0
Improved demo mode / obfuscation support 2025-04-21 21:49:41 -06:00
Matthew Holt
ec87974576
Refactor thumbnails jobs to dynamically page through rows by import ID 2025-04-21 16:18:23 -06:00
Matthew Holt
b88485a84b
A few fixes/enhancements
googlelocation: Allow iOS on-device location filename to be renamed, but it should still contain "location-history" and be a .json file.

- Upgrade mapbox-gl-js to 3.11

- Run thumbnail+embedding jobs even if import failed; WIP
2025-04-19 13:44:51 -06:00
Matthew Holt
b4d4f4b88e
Variety of minor UI enhancements 2025-04-13 22:51:30 -06:00
Matthew Holt
6d231fd0c2
Improved embeddings with SigLIP2; fix semantic search bug
Still lots of room for improvement here, but I see way better results already.
2025-04-13 15:38:14 -06:00
Matthew Holt
b4f43b662d
Super minor tweaks 2025-04-11 16:31:25 -06:00
Matthew Holt
a749c74747
Fix map size; auto-detect locale 2025-04-11 14:18:04 -06:00
Matthew Holt
758b10d6e5
Minor fixes for the conversations page 2025-04-11 12:21:20 -06:00
Matthew Holt
6ed4535933
Fix lint warning 2025-04-11 11:19:52 -06:00
Matthew Holt
70d8090c4e
ui: Add date picker to conversations view 2025-04-11 11:01:23 -06:00
Sergio Rubio
612cae9c03
Firefox data source (#55)
* [WiP] Firefox data source

Work in progress.

Implements a new Firefox datasource capable of reading its
places.sqlite database to import the browser history (page visits).

The implementation currently has a number of issues:

* Firefox (and Firefox based) browser keeps an exclusive lock on the
  places.sqlite database, and we can't dump or backup it while the
  browser is open, at least on Linux. To work around that,
  we copy the database to a temporary directory and import from it.
  This generally works, but isn't safe, as there's a risk of database
  corruption when doing the hot copy. Potential alternatives:
  * Ask the user to close the browser while the import happens, which
    isn't convenient/possible if this is happening regularly in the background.
  * Ignore and retry, as it'll eventually succeed, in the rare case the
    temporary db copy is corrupted and unreadable
  * Something else, no expert here.
* You need to point Timelinize to the places.sqlite file directly. Pointing
  it to the Firefox profile directory doesn't seem to work, as it
  doesn't seem to scan recursively or list all the directory files and
  pass them to Recognize. I'm probably missing something obvious here.
* Missing tests (will be added)

* Linter fixes

* Adapt it to the new API

* Send the full path to process

* Simplify import process

* Add datasource description

* Use the URL as the item content

* Add basic tests

* Give the test some more time

* Do not return an error if context was cancelled
2025-04-08 07:02:20 -06:00
Matthew Holt
73196f51ae
Refactor DirEntry, fix some bugs
Remove TopDir* functions, they aren't really relevant with our new import planner.
2025-04-02 21:52:49 -06:00
Matthew Holt
90e6ea228c
googlephotos: Fix empty imports
Thanks for the reports on Discord!
2025-04-01 22:04:33 -06:00
Matthew Holt
932831db47
Refactor data sources to make them dynamic
Also change the checkbox dropdown to a more interactive tomselect (type-to-search dropdown with chips) with pictures.

This makes it so data sources can be added to a timeline dynamically.

In the future, data sources can be implemented externally and push data to the timeline, so these need to not be rigidly hard-coded into the app and assumed to never change.

This essentially adds all their info (name, title, description, image, etc) into each timeline DB.
2025-02-11 16:49:20 -07:00
Matthew Holt
c20cf838d9
Fix data sources
Currently transitioning to letting them be dynamic
2025-02-07 13:08:55 -07:00
Matthew Holt
8c5d76dfad
Actually fix lint error 2025-02-07 12:08:25 -07:00
Matthew Holt
eb7717c843
Fix lint error; remove old build script
(Cross-compilation is now documented in our project wiki)
2025-02-07 12:02:47 -07:00
Matt Holt
35c5a63be4
Refactor python server code, update schema, rename config dir (#68)
* WIP

* Finish updating changes
2025-02-07 11:34:42 -07:00
Matthew Holt
d18c4cd2c8
googlelocation: Fix on-device Android format handling 2025-02-05 13:52:20 -07:00
Matthew Holt
c1a9abb74b
googlelocation: Support on-device Android 2025 format
(Thanks to those who helped in Discord!)
2025-01-30 13:08:26 -07:00
Matt Holt
628ecc1cb3
ci: Update workflows; restore functioning CI jobs (#64)
* ci: Attempt to fix broken CI

It broke out of the blue several months ago. I think ubuntu-latest
updated, but there's no PPA for libheif in that distro I guess

* Try tests next

* More fixing

* Try again

* Yada yada

* Woops

* I don't really know what I'm doing
2025-01-27 22:30:54 -07:00
Matthew Holt
7c34746b31
Minor fixes and enhancements; particularly to import planner 2025-01-27 06:54:44 -07:00
Matthew Holt
c3dc7728a1
Add basic calendar (.ics) data source 2025-01-26 14:39:02 -07:00
Matthew Holt
a6da2ee542
applecontacts: Split out this data source; imessage: FIxes & improvements 2025-01-22 22:35:46 -07:00
Matthew Holt
a4ec710f9f
Various fixes and improvements (imessage, duplicate rows, etc) 2025-01-21 10:16:38 -07:00
Matthew Holt
d76a211218
Refactor chart functions into one 2025-01-15 14:55:49 -07:00
Matthew Holt
7123e0d409
Replace ApexCharts with ECharts; UI fixes and improvements 2025-01-15 14:21:55 -07:00