Commit graph

314 commits

Author SHA1 Message Date
moneromooo-monero
83bbea4c7f
Add a is_key_image_spent daemon command and RPC call 2015-08-11 10:52:41 +01:00
moneromooo-monero
2634307863
daemon: omit extra set of <> in error message
The string conversion already adds them
2015-08-09 16:28:13 +01:00
moneromooo-monero
0822933524
daemon: print a decoded tx in print_tx
in addition to the raw hex representation
2015-08-09 16:25:05 +01:00
moneromooo-monero
1d678b1ee2
daemon: fix print_tx not find transactions
It was not even trying to
2015-08-09 15:48:02 +01:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
2e293a563e Fixed binary size issue due to embedded checkpoint data.
Fixed OSX compilation issues due to random lmdb resize points.
Fixed infinite loop bug when calling core::get_block_template(..).
2015-07-15 23:20:20 -07:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
e5d2680094 ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).

Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.

LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5

ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.

[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
   This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.

[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
	a. 0 = Compute long hash per block (may take a while depending on CPU)
	b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
	a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
	b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
	Fast    - Write meta-data but defer data flush.
	Fastest - Defer meta-data and data flush.
	Sync    - Flush data after nblocks_per_sync and wait.
	Async   - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
        Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
	Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
	For berkeley-db only. Auto remove logs if enabled.

**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
	At the moment, you need a full resync to use this optimized version.

[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.

Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block

**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop,   Dual-core / 4-threads U4200  (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).

lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop,  Quad-core / 8-threads 2600k  (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k  (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).

berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
2015-07-15 23:20:16 -07:00
moneromooo-monero
f9b361be8a
daemon: fix an exit crash 2015-06-14 00:31:09 +01:00
moneromooo-monero
35b160edea
fix ^D exit for bitmonerod
It uses the async console handler differently than simplewallet,
and wasn't running the same exit code, causing it to never actually
exit after breaking out of the console entry loop.
2015-06-03 22:43:35 +01:00
Riccardo Spagni
e01d32e52d
cleaning up, removing redundant files, renaming, fixing incorrect licenses 2015-05-31 13:40:18 +02:00
Riccardo Spagni
26fac96c5d
Merge pull request #294
693a8bf rpc_command_executor: fix mining in testnet mode (moneromooo-monero)
d09be11 core_rpc_server: add a getter for the testnet flag (moneromooo-monero)
754f863 core_rpc_server: log some error messages to the console (moneromooo-monero)
4f0e8cf daemon: remove obsolete daemon_commands_handler.h (moneromooo-monero)
bac366e core_rpc_server: reenable print_pl (moneromooo-monero)
39a9508 net_peerlist: fix grayt/white naming mismatch (moneromooo-monero)
8069b3b blockchain_db: add a few const (moneromooo-monero)
0932476 wallet: add consts where appropriate (moneromooo-monero)
2015-05-30 22:31:31 +02:00
moneromooo-monero
693a8bf7b1
rpc_command_executor: fix mining in testnet mode 2015-05-28 23:14:58 +01:00
moneromooo-monero
4f0e8cfa92
daemon: remove obsolete daemon_commands_handler.h 2015-05-28 14:10:46 +01:00
warptangent
10ff75e92e
daemon: Set log file default to use data dir
The log file previously used the default data dir even if --data-dir was
set to something else.

Document data dir and log file path.
2015-05-28 03:52:19 -07:00
moneromooo-monero
d8ee0a95c7
print limits when running limit commands with no arguments
It's more user friendly that an error message saying the command
does not exist.
2015-05-17 13:15:09 +01:00
warptangent
b76857f9d9
Add mempool output to daemon via command and RPC
This is for the "print_pool" command and "get_transaction_pool" RPC
method.

Add mempool's spent key images to the results.
2015-04-23 07:04:36 -07:00
rfree2monero
32c19c6c3d
[fix] log level change. compilation: dns, tests
old unbound #warning does not block compilation
unit tests build fine. Even though the RPC/P2P network type is required again
2015-04-10 16:54:21 +02:00
Thomas Winget
5b01493047
Fixes start_mining in cli interactive mode 2015-04-10 04:08:04 -04:00
Thomas Winget
a8bc7182ea
Merge BlockchainDB into upstream 2015-04-07 17:56:18 -04:00
rfree2monero
618f20ce49 Network 1.7; Quieted the debug a bit.
Really really finall version of this changes I hope.
2015-04-02 16:27:19 +02:00
rfree2monero
c511abf005 remerged; commands JSON. logging upgrade. doxygen 2015-04-01 19:00:45 +02:00
rfree2monero
3cbdf198f1 Merge remote-tracking branch 'monero-official/master' into network-1.6-work1 2015-04-01 18:24:45 +02:00
Thomas Winget
a0590d29cd
Restore daemon interactive mode
Daemon interactive mode is now working again.

RPC mapped calls in daemon and wallet have both had connection_context
removed as an argument as that argument was not being used anywhere.
2015-03-27 08:01:30 -04:00
Thomas Winget
7b14d4a17f
Steps toward multiple dbs available -- working
There will need to be some more refactoring for these changes to be
considered complete/correct, but for now it's working.

new daemon cli argument "--db-type", works for LMDB and BerkeleyDB.

A good deal of refactoring is also present in this commit, namely
Blockchain no longer instantiates BlockchainDB, but rather is passed a
pointer to an already-instantiated BlockchainDB on init().
2015-03-25 12:09:44 -04:00
Thomas Winget
5c0bc0050c
Merge upstream updates into blockchain branch 2015-03-25 05:56:36 -04:00
Thomas Winget
5eab480cb1
Moved BlockchainDB into its own src/ subfolder
Ostensibly janitorial work, but should be more relevant later down the
line.  Things that depend on core cryptonote things (i.e.
    cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
2015-03-06 15:20:45 -05:00
Thomas Winget
52f9629bd7
sending commands to forked daemon works on testnet now 2015-03-01 02:35:33 -05:00
Thomas Winget
76289d0e3b
Fix tests building -- function signatures changed 2015-03-01 02:21:19 -05:00
rfree2monero
f79821ac7e fix locking in count-peers thread (2) 2015-02-24 21:06:02 +01:00
rfree2monero
0198ffb220 2014 network limit 1.3 fix log/path/data +utils
+toc -doc -drmonero

Fixed the windows path, and improved logging and data
(for graph) logging, fixed some locks and added more checks.

Still there is a locking error,
not added by my patches, but present in master version
(locking of map/list of peers).
2015-02-24 20:12:56 +01:00
Thomas Winget
96cbecffd7
RPC calls for background daemon added in
The RPC calls the daemon executable uses to talk to the running daemon
instance have mostly been added back in.  Rate limiting has not been
added in upstream, but is on its way in a separate effort, so those
calls are still NOPed out.
2015-02-24 00:44:36 -05:00
Thomas Winget
9193d6fb5b
Daemonize changes pulled in -- daemon builds
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return).  Other than that, this *should*
work...at any rate, it builds, and that's something.
2015-02-24 00:05:19 -05:00
rfree2monero
ae2a50659f 2014 network limit 1.2 +utils +toc -doc -drmonero
new update of the pr with network limits

more debug options:
discarding downloaded blocks all or after given height.
trying to trigger the locking errors.

debug levels polished/tuned to sane values.
debug/logging improved.

warning: this pr should be correct code, but it could make
an existing (in master version) locking error appear more often.

it's a race on the list (map) of peers, e.g. between closing/deleting
them versus working on them in net-limit sleep in sending chunk.

the bug is not in this code/this pr, but in the master version.

the locking problem of master will be fixed in other pr.

problem is ub, and in practice is seems to usually cause program abort
(tested on debian stable with updated gcc). see --help for option
to add sleep to trigger the error faster.
2015-02-20 22:28:03 +01:00
rfree2monero
5ce4256e3d 2014 network limit 1.1 +utils +toc -doc -drmonero
Update of the PR with network limits

works very well for all speeds
(but remember that low download speed can stop upload
because we then slow down downloading of blockchain
requests too)

more debug options

fixed pedantic warnings in our code
should work again on Mac OS X and FreeBSD
fixed warning about size_t
tested on Debian, Ubuntu, Windows(testing now)

TCP options and ToS (QoS) flag
FIXED peer number limit
FIXED some spikes in ingress/download
FIXED problems when other up and down limit
2015-02-20 22:13:00 +01:00
rfree2monero
eabb519605 2014 network limit 1.0a +utils +toc -doc -drmonero
commands and options for network limiting
works very well e.g. for 50 KiB/sec up and down
ToS (QoS) flag
peer number limit
TODO some spikes in ingress/download
TODO problems when other up and down limit
added "otshell utils" - simple logging (with colors, text files channels)
2015-02-20 22:13:00 +01:00
Riccardo Spagni
f4b69d553a
year updated in license 2015-01-02 18:52:46 +02:00
Ben Boeckel
01895dd067 cmake: fix up link lines 2014-10-24 15:30:57 -04:00
Ben Boeckel
7d708e4223 cmake: support 2.8.7
Older versions of CMake support LINK_{PUBLIC,PRIVATE} while newer
versions prefer PUBLIC and PRIVATE instead, but still support the LINK_
prefix.
2014-10-24 15:29:51 -04:00
Ben Boeckel
f53f04724c cmake: handle private vs. public headers 2014-10-23 16:42:34 -04:00
Ben Boeckel
c773f465ca cmake: refactor common code with executables 2014-10-23 16:42:34 -04:00
Ben Boeckel
89cff7bddc cmake: put each library into its own directory
This cleans up the CMake code and shows patterns more easily (to be
refactored in the next commit).
2014-10-23 16:42:34 -04:00
Thomas Winget
7568f89c55 Fixed segfault with checkpoints loading
Bounds checking on blockchain_storage' m_blocks.size() when validating
against checkpoints.  Also moved initial json & DNS checkpoints load to
after blockchain init.
2014-09-30 16:21:37 -04:00
Thomas Winget
b261d9207b DNS checkpoint updating added, and daemon flag to enforce them
The daemon should now check for updated checkpoints from
checkpoints.moneropulse.org as well as from the configured json file
every ~1hr (and on launch).

The daemon now has a flag to enable enforcing these checkpoints (rather
    than just printing a warning when they fail).

TODO: an easily configurable list of DNS servers to check for
checkpoints as opposed to the hard-coded "checkpoints.moneropulse.org"
2014-09-30 16:21:37 -04:00
Thomas Winget
30caebfce3 reload checkpoints file every ~hr and print if any fail
also some other minor bug squashing and code formatting
2014-09-30 16:21:37 -04:00
Tomer Konforty
06a4578bf2 Added ability to read chechpoint hashes from json file in data folder 2014-09-25 00:00:44 +02:00
Zachary Michaels
59ab569da1 Give up on brace initializers in initializer lists (MSVC bug) 2014-09-15 23:14:24 +02:00
Riccardo Spagni
72a80f6213
fixed incorrect version reference 2014-09-15 15:58:22 +02:00
Zachary Michaels
d03308734b
Separate testnet address prefix 2014-09-15 15:54:59 +02:00
Zachary Michaels
4a6eb0a016 Create testnet data dir if necessary 2014-09-15 15:54:24 +02:00
Zachary Michaels
018e251cc0
Separate testnet default data dir 2014-09-15 15:54:19 +02:00
Zachary Michaels
257077a96b Separate network id for testnet 2014-09-15 15:53:50 +02:00
Zachary Michaels
658b6690a3
Separate rpc port for testnet 2014-09-15 15:53:46 +02:00
Zachary Michaels
07470fd400
Add testnet flag
Source: cryptonotefoundation
2014-09-15 15:53:01 +02:00
Riccardo Spagni
8334ce0b24 versioning now includes the commit hash, or -final for tagged releases 2014-09-12 13:12:43 +02:00
jebes
dfed3d39b6 Merge remote-tracking branch 'upstream/master' 2014-09-11 10:20:55 -04:00
jebes
c085e9294f commented util.h 2014-09-09 20:18:23 -04:00
fluffypony
cf5a8b1d6c moved non-critical warnings and errors to log level 1 2014-09-09 11:32:00 +02:00
fluffypony
6fc995fe5d License updated to BSD 3-clause 2014-07-23 15:03:52 +02:00
mydesktop
3a3a817678 0.8.8update 2014-05-25 13:06:40 -04:00
NoodleDoodleNoodleDoodleNoodleDoodleNoo
a03b13b50b Update daemon_commands_handler.h
1. Added "diff" command to show current estimated difficulty and hash-rate.
2014-05-13 17:55:50 -07:00
mydesktop
79a4bedc36 mac osx building fixes 2014-04-30 16:50:06 -04:00
Antonio Juarez
a401a02ddb Improvements in JSON RPC 2014-04-07 16:02:15 +01:00
Antonio Juarez
29c2859a3e json rpc for wallet and bugfix 2014-04-02 17:00:17 +01:00
Antonio Juarez
8efa1313f3 some fixes 2014-03-20 11:46:11 +00:00
Antonio Juarez
296ae46ed8 moved all stuff to github 2014-03-03 22:07:58 +00:00