1
0
Fork 0
Commit graph

162 commits

Author SHA1 Message Date
Matthew Holt
cce63835c5 Add toggle for hidden files in file picker (close #69) 2025-04-09 07:50:38 -06:00
Sergio Rubio
612cae9c03
Firefox data source (#55)
* [WiP] Firefox data source

Work in progress.

Implements a new Firefox datasource capable of reading its
places.sqlite database to import the browser history (page visits).

The implementation currently has a number of issues:

* Firefox (and Firefox based) browser keeps an exclusive lock on the
  places.sqlite database, and we can't dump or backup it while the
  browser is open, at least on Linux. To work around that,
  we copy the database to a temporary directory and import from it.
  This generally works, but isn't safe, as there's a risk of database
  corruption when doing the hot copy. Potential alternatives:
  * Ask the user to close the browser while the import happens, which
    isn't convenient/possible if this is happening regularly in the background.
  * Ignore and retry, as it'll eventually succeed, in the rare case the
    temporary db copy is corrupted and unreadable
  * Something else, no expert here.
* You need to point Timelinize to the places.sqlite file directly. Pointing
  it to the Firefox profile directory doesn't seem to work, as it
  doesn't seem to scan recursively or list all the directory files and
  pass them to Recognize. I'm probably missing something obvious here.
* Missing tests (will be added)

* Linter fixes

* Adapt it to the new API

* Send the full path to process

* Simplify import process

* Add datasource description

* Use the URL as the item content

* Add basic tests

* Give the test some more time

* Do not return an error if context was cancelled
2025-04-08 07:02:20 -06:00
Matthew Holt
73196f51ae
Refactor DirEntry, fix some bugs
Remove TopDir* functions, they aren't really relevant with our new import planner.
2025-04-02 21:52:49 -06:00
Matthew Holt
932831db47
Refactor data sources to make them dynamic
Also change the checkbox dropdown to a more interactive tomselect (type-to-search dropdown with chips) with pictures.

This makes it so data sources can be added to a timeline dynamically.

In the future, data sources can be implemented externally and push data to the timeline, so these need to not be rigidly hard-coded into the app and assumed to never change.

This essentially adds all their info (name, title, description, image, etc) into each timeline DB.
2025-02-11 16:49:20 -07:00
Matthew Holt
c20cf838d9
Fix data sources
Currently transitioning to letting them be dynamic
2025-02-07 13:08:55 -07:00
Matthew Holt
c3dc7728a1
Add basic calendar (.ics) data source 2025-01-26 14:39:02 -07:00
Matthew Holt
a6da2ee542
applecontacts: Split out this data source; imessage: FIxes & improvements 2025-01-22 22:35:46 -07:00
Matthew Holt
ccef13f530
Add icons for iCloud and iMessage 2025-01-21 15:53:02 -07:00
Matthew Holt
a4ec710f9f
Various fixes and improvements (imessage, duplicate rows, etc) 2025-01-21 10:16:38 -07:00
Matthew Holt
83635f444d
WIP entity page 2025-01-16 14:43:46 -07:00
Matthew Holt
7123e0d409
Replace ApexCharts with ECharts; UI fixes and improvements 2025-01-15 14:21:55 -07:00
Matthew Holt
4e89fca643
Fix relationship de-duping; speed up imports a bit more 2025-01-10 15:31:20 -07:00
Matthew Holt
29e2bc8fef
Fix iphone/imessage: Update attribute_id in DB if inserting item piecewise
iMessage db may send a reaction graph for a message before sending the message itself to the pipeline, thus an empty item with only an original ID gets inserted, and later the full message item comes in, but I had neglected to add attribute_id to updateOverrides.
2025-01-09 18:09:58 -07:00
Matthew Holt
bfbe2e5f8f Minor fixes for empty timelines 2025-01-09 05:36:15 -07:00
Matthew Holt
8b36413457
Upgrade Tabler to beta 23; minor fix on Entity page 2025-01-08 12:38:47 -07:00
Matthew Holt
e5626ec9ab
Quick fix of conversations filter 2025-01-07 13:52:55 -07:00
Matthew Holt
bb9151628f
WIP: new entity page; delete almost all DB indexes
Imports are now 4-5x faster and queries are still just about as fast. New indexes should only be created after proving their usefulness.
2025-01-07 13:42:05 -07:00
Matthew Holt
5ec783ecb8
Coupla fixes; also file picker now remembers last path 2025-01-02 20:33:50 -07:00
Matthew Holt
8c573eb64c
Use integer radius for obfuscated locations to match backend int type 2025-01-02 14:11:06 -07:00
Matthew Holt
5fa0b5b3b7
Enhance $() to make it more flexible
It can now accept a selector string or a DOM element
2025-01-02 13:38:03 -07:00
Matthew Holt
a00e478f32
Fix entity select in conversations view 2025-01-02 13:12:05 -07:00
Matthew Holt
ec5c6eb479
Upgrade Tabler to beta 22; new chat UI for messages 2025-01-02 11:26:35 -07:00
Matthew Holt
9a78d60ef9
Fix some UI filter related issues 2025-01-02 08:10:15 -07:00
Matthew Holt
b68b917de4
Fix bug wiping out obfuscated locations 2025-01-01 17:32:33 -07:00
Matthew Holt
aea113edc4
Ability to set own Mapbox API key (closes #5)
Required some refactoring; settings are now loaded when SPA loads, and are updated when changed.
2025-01-01 17:24:07 -07:00
Matthew Holt
35979b4081
WIP: Config change that restarts app; refactor, bug fix map draw 2025-01-01 16:31:09 -07:00
Matthew Holt
5816c571c6
WIP settings: save is starting to function 2024-12-31 20:58:07 -07:00
Matthew Holt
4bd08bd91c
WIP settings endpoints; use attr as alternate display name in messages 2024-12-31 10:09:20 -07:00
Matthew Holt
a1577ca24a
Upgrade to Tabler beta 21, with fixes for breaking changes 2024-12-30 14:17:37 -07:00
Matthew Holt
3d11d65b8d
WIP settings page; #map mobility; WIP interactive imports
Settings page is started; non-functional, but location picker works.

Moving maps between container elements is improved by moving to nearest to mouse pointer, rather than just most center to the viewport. It also emits an event when the map is moved, allowing us to change/reset map configurations for certain displays.

More progress on interactive imports. More thought is needed before continuing.

Upgraded Mapbox libraries.
2024-12-26 11:51:47 -07:00
Matthew Holt
ce297389b0
Thumbnail job streaming; WIP: interactive imports 2024-12-19 06:51:06 -07:00
Matthew Holt
294e2a72a9
Reconnect after disconnection; improve checkpointing 2024-12-17 14:27:50 -07:00
Matthew Holt
d7b1d73796
Auto-resume jobs on start; improve checkpoint performance 2024-12-16 16:23:26 -07:00
Matthew Holt
e319e1f60d
Checkpoints for Google Location; minor jobs fix 2024-12-16 10:57:34 -07:00
Matthew Holt
a4d8bc923d
Data source checkpoints; refine import concurrency
And related improvements and fixes
2024-12-15 22:40:58 -07:00
Matthew Holt
024e9a4622
Implement job start/restart 2024-12-13 14:30:13 -07:00
Matthew Holt
fcaa238634
Implement pause/unpause 2024-12-13 13:02:06 -07:00
Matthew Holt
22628833a7
Refactor obfuscation mode and some processing logic 2024-12-13 07:19:27 -07:00
Matthew Holt
0063bbe396
Some fixes for import streaming 2024-12-12 10:37:51 -07:00
Matthew Holt
786f516696
Refine import stream 2024-12-12 10:18:28 -07:00
Matthew Holt
742a543977
WIP live stream of imported items 2024-12-12 08:26:37 -07:00
Matthew Holt
fa7efca355
Minor job UI tweaks 2024-12-11 17:34:27 -07:00
Matthew Holt
27e314e347
Few more minor jobs fixes, tunings 2024-12-11 15:41:13 -07:00
Matthew Holt
b4149dc649
More live job stats/rendering improvements 2024-12-11 14:04:07 -07:00
Matthew Holt
687fb66c47
Improve job throughput display 2024-12-11 09:58:42 -07:00
Matthew Holt
274397afe2
Fix negative durations in job status 2024-12-11 06:01:01 -07:00
Matthew Holt
aa12d85c22
Fix job cancellation; wire up more of job UI 2024-12-10 23:13:14 -07:00
Matthew Holt
9ce1efa117
WIP live view of active jobs 2024-12-09 21:55:44 -07:00
Matthew Holt
063501c0f9
Fix stuck thumbnail loaders 2024-12-08 08:03:20 -07:00
Matthew Holt
13131aba65
Run ANALYZE after imports and at startup; add NMEA icon 2024-12-08 05:29:17 -07:00