- Somehow I totally forgot to relate sidecar motion photos in Google Photos. (They don't use sidecars on Google phones.)
- Item page now displays entities in the picture even without face coordinates
- Quick unit tests for a function related to Google Takeout archives
- We now combine existing metadata with new according to the update policy, instead of either writing all or none of incoming metadata. This merging happens before the DB update query and is a bit of a special case as the policy is applied per-key.
- Special handling for corrupted timestamp in Google Photos data. This is a singular case I haven't observed more of, but seems like a reasonable heuristic. There might be thousands more out there, who knows.
- Fix job creation time (milliseconds)
- Hopefully make repeated imports faster by skipping duplicate items more intelligently based on update policies.
This is useful on the gallery page where we do NOT want to show motion pictures. We will also need to block motion pictures from being displayed as separate items on other UI views when they do show non-root items.
The sorting can help imports go faster if we put DB-heavy sources first, when the database is still small.
The data source names were also standardized to use snake_case like most other word-IDs in the app.
- Obfuscation mode enabled would set a fake phone number in smsbackuprestore's DS options, which led to bad data. Now, the UI does not auto-fill that value. But that means we need...
- SMS Backup & Restore: Phone number can now be inferred from repo owner in the backend, if ds opt phone number is empty. This works even with obfuscation enabled.
- Aborting a scheduled job before it starts now stays aborted. (Unless you manually restart it.)
- Added a data validation error modal for DS options on the import page. For now, if smsbackuprestore has no phone number set, and the timeline repo owner doesn't have a phone number, an error will be shown.
- Timestamp year cannot be > 9999 (JSON serialization panics)
- Lat/lon now considered equivalent after a certain decimal point, since not all sources have high precision (we choose 5 decimal points for now, or about 1.1 meters)
- Map style must be loaded before source is added, apparently (got this error once)
The gofakeit upgrade uses the new math/rand/v2 package, which uses uint64 more than int64, so we had to change a bunch of row IDs from int64 to uint64.
googlelocation: Allow iOS on-device location filename to be renamed, but it should still contain "location-history" and be a .json file.
- Upgrade mapbox-gl-js to 3.11
- Run thumbnail+embedding jobs even if import failed; WIP
* [WiP] Firefox data source
Work in progress.
Implements a new Firefox datasource capable of reading its
places.sqlite database to import the browser history (page visits).
The implementation currently has a number of issues:
* Firefox (and Firefox based) browser keeps an exclusive lock on the
places.sqlite database, and we can't dump or backup it while the
browser is open, at least on Linux. To work around that,
we copy the database to a temporary directory and import from it.
This generally works, but isn't safe, as there's a risk of database
corruption when doing the hot copy. Potential alternatives:
* Ask the user to close the browser while the import happens, which
isn't convenient/possible if this is happening regularly in the background.
* Ignore and retry, as it'll eventually succeed, in the rare case the
temporary db copy is corrupted and unreadable
* Something else, no expert here.
* You need to point Timelinize to the places.sqlite file directly. Pointing
it to the Firefox profile directory doesn't seem to work, as it
doesn't seem to scan recursively or list all the directory files and
pass them to Recognize. I'm probably missing something obvious here.
* Missing tests (will be added)
* Linter fixes
* Adapt it to the new API
* Send the full path to process
* Simplify import process
* Add datasource description
* Use the URL as the item content
* Add basic tests
* Give the test some more time
* Do not return an error if context was cancelled
Also change the checkbox dropdown to a more interactive tomselect (type-to-search dropdown with chips) with pictures.
This makes it so data sources can be added to a timeline dynamically.
In the future, data sources can be implemented externally and push data to the timeline, so these need to not be rigidly hard-coded into the app and assumed to never change.
This essentially adds all their info (name, title, description, image, etc) into each timeline DB.
