iMessage db may send a reaction graph for a message before sending the message itself to the pipeline, thus an empty item with only an original ID gets inserted, and later the full message item comes in, but I had neglected to add attribute_id to updateOverrides.
fs.SkipDir documentation is a bit unclear: does it skip the remainder of files in the directory when returned from walking a file, or does it no-op on files and only skip going INTO dirs when on a dir?
I thought it was the latter, and thus, we didn't need to check whether the current DirEntry was a directory before returning (most commonly, when we are trying to skip hidden files/folders). But nope, it's the former -- SkipDir will skip the rest of the entries in the directory, which is NOT what we want. We just want to avoid going INTO a hidden directory in our case.
So unfortunately we now have to check IsDir() before returning.
Also fixed a slight bug with NMEA processing.
* Schema revisions for new import flow and thumbnails
* WIP settings
* WIP quick schema fix
* gallery: Image search using ML embeddings
Still very rough around the edges, but basically works.
'uv' gets auto-installed, but currently requires restarting Timelinize before it can be used.
Lots of tunings and optimizations are needed. There is much room for improvement.
Still migrating from imports -> jobs, so that part of the code and schema is still a mess.
* Implement search for similar items
* Finish import/planning rewrite; it compiles and tests pass
* Fix some bugs, probably introduce other bugs
* WIP new import planning page
* Fix Google Photos and Twitter recognition
* Finish most of import page UI; start button still WIP
* WIP: Start Import button
* Fixes to jobs, thumbnail job, import job, etc.
* Implement proper checkpointing support; jobs fixes
* Invalid file path when importing a single file
Adds a test that proves importing a single file sets the file name to
'.'
* Clean test a bit
* Expand test
* 💄 test
* One more test
* Appease the linter
* 💄
* Add missing fixture
* Set the right Filepath when processing single files
* Simpler fix suggested by @mholt
* Simplify tests
* Add clarifying comment
* Rename the test
* GitHub stars data source
Data source that imports GitHub starred repositories in JSON format.
Each starred repo is imported individually, the starred repo
metadata comes from the GitHub API.
The item timestamp is set to the starred date, so they appear in the
timeline the day the repo was starred.
A small json file is saved in the timeline repository data directory
with the metadata retrieved from the GitHub API, which looks like:
```
{
"id": 841044067,
"name": "timelinize",
"html_url": "https://github.com/timelinize/timelinize",
"description": "Store your data from all your accounts and devices in a single cohesive timeline on your own computer",
"created_at": "2024-08-11T13:27:39Z",
"updated_at": "2024-09-03T07:17:29Z",
"pushed_at": "2024-09-02T15:31:59Z",
"stargazers_count": 504,
"language": "Go",
"full_name": "timelinize/timelinize",
"topics": null,
"is_template": false,
"Topics": "archival,data-archiving,data-import,timeline",
"private": false,
"starred_at": "2024-08-12T17:55:48Z"
}
```
The data source currently expects the JSON file to be named like:
- ghstars.json
- ghstars-<ISO date>.json
- ghstars-<UNIX timestamp>.json
* Linter fixes
* Remove optional options
* Add the URL label to the bookmark class
* Change the data source name to GitHub
* Rename data source directory also
* Rename datasource main file
* Store GitHub starred repo URL only
* rename symbols
* Add basic tests
* moar tests
* Linter fix
* You can read on closed channels
* Add bookmark svg for the frontend
* Update package docs
* 💄 docs
* Update datasources/github/github.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Update datasources/github/github.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Update datasources/github/github.go
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* Remove content from item
---------
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* chore: add workflow to run go test
* fix a test failure
* Downgrade heif to be able to test
* run all tests by default
* Revert typo fix
* Fix metadata test
* Accept geojson files without a time property: do not return an error
if well-know time properties aren't found, which prevents the entire
feature from being ingested.
* Random properties aren't stored sometimes: do not modify the map while
iterating it, as that can have unpredictable effects according to the spec
(https://go.dev/ref/spec#For_range).
Certain property names related to time, altitude, heading, etc., are extracted to the best of our ability.
Updates #33
Also a minor fix for a bug in spa.js when creating a new timeline with no owner info (other than name).
Stores additiona feature properties as metadata, which makes for a nicer
rendered object:
- Name: name given to the feature
- Country: readable country name
- ISO country code
- Notes: arbitrary text notes
The spec[^1] doesn't seem to define the properties allowed here, it can
be an arbitrary JSON object, so maybe a better option in the future
would be to read and save all the properties available.
[^1]: https://datatracker.ietf.org/doc/html/rfc7946#section-3.2
