Commit graph

359 commits

Author SHA1 Message Date
Riccardo Spagni
9a4b15c631
Merge pull request #4664
61304151 db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedup (moneromooo-monero)
2018-11-01 22:34:20 +02:00
Riccardo Spagni
4f7cdd90fd
Merge pull request #4639
ec301390 Update db_lmdb.cpp (Cactii1)
2018-11-01 22:24:11 +02:00
moneromooo-monero
99fbe1008b
db_lmdb: save some string copies for readonly db keys/values 2018-11-01 17:26:09 +00:00
moneromooo-monero
6ecc99ad1f
core: avoid unnecessary tx/blob conversions 2018-11-01 15:51:19 +00:00
moneromooo-monero
61304151b4
db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedup 2018-10-19 16:47:29 +00:00
Cactii1
ec3013903b
Update db_lmdb.cpp 2018-10-17 21:36:10 +02:00
moneromooo-monero
4eca42b241
blockchain_db: initialize m_hardfork in ctor just in case
Coverity 136568
2018-10-16 11:58:46 +00:00
Howard Chu
7c790f11f2
Fix rtxn usage in BlockchainLMDB::get_estimated_batch_size
Should only stop the rtxn if we actually started it
Fixes Coverity 184960
2018-10-02 16:14:20 +01:00
Riccardo Spagni
effcbf2060
Merge pull request #4459
bcf3f6af fuzz_tests: catch unhandled exceptions (moneromooo-monero)
3ebd05d4 miner: restore stream flags after changing them (moneromooo-monero)
a093092e levin_protocol_handler_async: do not propagate exception through dtor (moneromooo-monero)
1eebb82b net_helper: do not propagate exceptions through dtor (moneromooo-monero)
fb6a3630 miner: do not propagate exceptions through dtor (moneromooo-monero)
2e2139ff epee: do not propagate exception through dtor (moneromooo-monero)
0749a8bd db_lmdb: do not propagate exceptions in dtor (moneromooo-monero)
1b0afeeb wallet_rpc_server: exit cleanly on unhandled exceptions (moneromooo-monero)
418a9936 unit_tests: catch unhandled exceptions (moneromooo-monero)
ea7f9543 threadpool: do not propagate exceptions through the dtor (moneromooo-monero)
6e855422 gen_multisig: nice exit on unhandled exception (moneromooo-monero)
53df2deb db_lmdb: catch error in mdb_stat calls during migration (moneromooo-monero)
e67016dd blockchain_blackball: catch failure to commit db transaction (moneromooo-monero)
661439f4 mlog: don't remove old logs if we failed to rename the current file (moneromooo-monero)
5fdcda50 easylogging++: test for NULL before dereference (moneromooo-monero)
7ece1550 performance_test: fix bad last argument calling add_arg (moneromooo-monero)
a085da32 unit_tests: add check for page size > 0 before dividing (moneromooo-monero)
d8b1ec8b unit_tests: use std::shared_ptr to shut coverity up about leaks (moneromooo-monero)
02563bf4 simplewallet: top level exception catcher to print nicer messages (moneromooo-monero)
c57a65b2 blockchain_blackball: fix shift range for 32 bit archs (moneromooo-monero)
2018-09-29 22:20:38 +02:00
Riccardo Spagni
09fb2bb5f8
Merge pull request #4445
fe125647 Fixup RENAME_DB() macro (Howard Chu)
2018-09-29 22:20:12 +02:00
moneromooo-monero
befdcbf4be
db_lmdb: do not use base for cumulative distribution
it's confusing and needlessly complicated
2018-09-12 19:20:13 +00:00
moneromooo-monero
5ffb2ff9b7
v8: per byte fee, pad bulletproofs, fixed 11 ring size 2018-09-11 13:38:07 +00:00
luigi1111
ab85b924c9
Merge pull request #4293
9d65399 is_hdd update (p8p)
2018-09-10 15:07:02 -05:00
luigi1111
46bcd91db4
Merge pull request #4256
d6fc870 db_lmdb: resize blockchain database when 90% filled (moneromooo-monero)
2018-09-10 15:04:50 -05:00
luigi1111
77ed11e627
Merge pull request #4270
29dea03 epee: resize vectors where possible in serialization (moneromooo-monero)
76affd9 epee: some speedup in parsing (moneromooo-monero)
dc6c069 db_lmdb: speedup the get_output_distribution common case (moneromooo-monero)
76ac5a8 wallet2: ask for a binary output distribution, for speed (moneromooo-monero)
2018-09-04 13:19:58 -05:00
p8p
9d6539923e
is_hdd update 2018-08-25 04:31:22 -07:00
luigi1111
4a4ea78ecd
Merge pull request #4204
b278b83 core: sync database based on bytes added, not blocks added (moneromooo-monero)
2018-08-22 20:43:51 -05:00
luigi1111
7e4507a048
Merge pull request #4200
43f7110 blockchain_db: remove unused get_output_key variant (moneromooo-monero)
2018-08-22 20:40:37 -05:00
moneromooo-monero
dc6c0696fd
db_lmdb: speedup the get_output_distribution common case 2018-08-19 15:58:21 +00:00
luigi1111
2530dc6710
Merge pull request #4161
be02eb9 db_lmdb: demote the 'batch already enabled' log, it's harmless (moneromooo-monero)
2018-08-15 17:27:48 -05:00
moneromooo-monero
d6fc870d04
db_lmdb: resize blockchain database when 90% filled
instead of a random ratio from 60% to 90%.
2018-08-14 08:55:49 +00:00
moneromooo-monero
b278b83860
core: sync database based on bytes added, not blocks added
Blocks have a very wide range, whereas actual size is the relevant
quantity to consider when syncing
2018-08-12 16:13:46 +00:00
moneromooo-monero
43f71100bc
blockchain_db: remove unused get_output_key variant
It was actually incorrect, as it would not return commitment
2018-07-30 21:57:47 +00:00
luigi1111
0dddfeacc9
Merge pull request #4129
ee31383 db_lmdb: don't sync a read only DB (moneromooo-monero)
2018-07-27 14:30:57 -05:00
moneromooo-monero
be02eb9389
db_lmdb: demote the "batch already enabled" log, it's harmless 2018-07-20 23:41:40 +01:00
luigi1111
3e026ff6ed
Merge pull request #4013
e5592c4 rpc: add blockchain disk size to getinfo (moneromooo-monero)
2018-07-19 13:40:42 -05:00
luigi1111
21afa71ba6
Merge pull request #3981
45e419b db: store cumulative rct output distribution in the db for speed (moneromooo-monero)
2018-07-19 13:39:24 -05:00
luigi1111
025187e6c9
Merge pull request #3854
149da42 db_lmdb: enable batch transactions by default (stoffu)
34cb6b4 add --regtest and --fixed-difficulty for regression testing (vicsn)
9e1403e update get_info RPC and bump RPC version (vicsn)
207b66e first new functional tests (vicsn)
2018-07-19 13:35:25 -05:00
moneromooo-monero
45e419bd5c
db: store cumulative rct output distribution in the db for speed
This gets rid of the temporary precalc cache.

Also make the RPC able to send data back in binary or JSON,
since there can be a lot of data

This bumps the LMDB database format to v3, with migration.
2018-07-13 11:37:04 +01:00
moneromooo-monero
ee31383a52
db_lmdb: don't sync a read only DB
This would only throw
2018-07-11 18:14:22 +01:00
victorsintnicolaas
34cb6b4b70 add --regtest and --fixed-difficulty for regression testing
on_generateblocks RPC call combines functionality from the on_getblocktemplate and on_submitblock RPC calls to allow rapid block creation. Difficulty is set permanently to 1 for regtest.
Makes use of FAKECHAIN network type, but takes hard fork heights from mainchain
Default reserve_size in generate_blocks RPC call is now 1. If it is 0, the following error occurs 'Failed to calculate offset for'.
Queries hard fork heights info of other network types
2018-06-29 10:06:11 +02:00
luigi1111
cbfa0a2a7e
Merge pull request #3987
2d10830 db_ldmb: print db version when complaining about incompatibility (moneromooo-monero)
2018-06-27 16:23:17 -05:00
luigi1111
c3ec5373b3
Merge pull request #3936
d81e042 tx_pool: initialize bitflags padding since it gets written to storage (moneromooo-monero)
2018-06-25 14:53:27 -05:00
moneromooo-monero
e5592c4bab
rpc: add blockchain disk size to getinfo
This should help new nodes predict how much disk space will be
needed for a full sync
2018-06-20 23:48:37 +01:00
stoffu
149da420e9 db_lmdb: enable batch transactions by default 2018-06-14 20:56:11 +02:00
moneromooo-monero
2d10830cfb
db_ldmb: print db version when complaining about incompatibility 2018-06-11 17:57:02 +01:00
moneromooo-monero
284fe6438d
db_lmdb: warn about slowness when running off a spinning disk 2018-06-08 20:59:02 +01:00
moneromooo-monero
d81e042306
tx_pool: initialize bitflags padding since it gets written to storage
Avoids valgrind reporting uninitialized data usage
2018-06-05 20:26:44 +01:00
luigi1111
a7ea14dc6a
Merge pull request #3876
740da1b core: fix automatic safe db sync mode switching (moneromooo-monero)
e942d34 protocol: do not switch to unsafe sync mode for just a few blocks (moneromooo-monero)
2018-05-30 15:46:13 -05:00
moneromooo-monero
740da1ba95
core: fix automatic safe db sync mode switching 2018-05-28 12:14:04 +01:00
moneromooo-monero
b9389e582e
db_lmdb: save pruned and prunable tx data separately
This bumps DB version to 2, migration code will run for v1 DBs
2018-05-23 22:48:12 +01:00
moneromooo-monero
5f8ffca34d
speed up get_output_distribution (and precalc common case) 2018-05-21 19:51:58 +01:00
moneromooo-monero
ffeeefde60
speedup get_output_histogram for all amounts when min_count > 0
This skips the vast majority of "dust" output amounts with just
one instance on the chain. Clocks in at 0.15% of the original
time on testnet.
2018-04-23 10:54:28 +01:00
moneromooo-monero
41f727ce42
add RPC to get a histogram of outputs of a given amount 2018-03-16 10:32:26 +00:00
Riccardo Spagni
0e5c006207
Merge pull request #3307
223d7d0c db_lmdb: fix free space reporting (moneromooo-monero)
2018-03-05 19:11:52 +02:00
Edward Betts
fbcc91c2a4 Correct spelling mistakes. 2018-03-05 17:00:40 +00:00
moneromooo-monero
223d7d0c7a
db_lmdb: fix free space reporting
reported by Brad Richards
2018-02-23 09:06:10 +00:00
moneromooo-monero
6514f0ed7b
db_lmdb: fix return code mixup in for_all_* 2018-02-19 14:55:58 +00:00
Riccardo Spagni
f4a6bc79d9
Merge pull request #3226
e4646379 keccak: fix mdlen bounds sanity checking (moneromooo-monero)
2e3e90ac pass large parameters by const ref, not value (moneromooo-monero)
61defd89 blockchain: sanity check number of precomputed hash of hash blocks (moneromooo-monero)
9af6b2d1 ringct: fix infinite loop in unused h2b function (moneromooo-monero)
8cea8d0c simplewallet: double check a new multisig wallet is multisig (moneromooo-monero)
9b98a6ac threadpool: catch exceptions in dtor, to avoid terminate (moneromooo-monero)
24803ed9 blockchain_export: fix buffer overflow in exporter (moneromooo-monero)
f3f7da62 perf_timer: rewrite to make it clear there is no division by zero (moneromooo-monero)
c6ea3df0 performance_tests: remove add_arg call stray extra param (moneromooo-monero)
fa6b4566 fuzz_tests: fix an uninitialized var in setup (moneromooo-monero)
03887f11 keccak: fix sanity check bounds test (moneromooo-monero)
ad11db91 blockchain_db: initialize m_open in base class ctor (moneromooo-monero)
bece67f9 miner: restore std::cout precision after modification (moneromooo-monero)
1aabd14c db_lmdb: check hard fork info drop succeeded (moneromooo-monero)
2018-02-16 14:26:58 +01:00
Riccardo Spagni
260d666cf4
Merge pull request #3231
84a8b2da Don't create readtxn until after txn_safe gate check (Howard Chu)
2018-02-16 14:24:01 +01:00
moneromooo-monero
ad11db9144
blockchain_db: initialize m_open in base class ctor
It's cleaner this way, since it's a base class field

Coverity 136568
2018-02-02 15:21:39 +00:00
moneromooo-monero
1aabd14c21
db_lmdb: check hard fork info drop succeeded
Coverity 136364
2018-02-02 15:08:22 +00:00
Howard Chu
84a8b2da14
Don't create readtxn until after txn_safe gate check 2018-01-31 18:18:01 +00:00
xmr-eric
18216f19dd Update 2018 copyright 2018-01-26 10:03:20 -05:00
Riccardo Spagni
0a110aefbb
Merge pull request #2971
ae860230 Fix exceptions not finding txpool txes when relaying (moneromooo-monero)
2018-01-10 11:45:15 +01:00
Howard Chu
294adc8341
Additional fix for core_tests
Reset thread-local info if it doesn't match the current env.
Only happens when a process opens/closes env multiple times in the
same process, doesn't affect monerod.
2017-12-28 20:24:08 +00:00
Howard Chu
d52b732efb
Fix stale readcursor flags
Reset thread-specific flags when a write txn is started.
Also remove some redundant start-readtxn code.
2017-12-28 00:59:04 +00:00
moneromooo-monero
ae8602303a
Fix exceptions not finding txpool txes when relaying 2017-12-20 13:02:53 +00:00
moneromooo-monero
a4240d9ffc
catch const exceptions 2017-12-18 15:15:43 +00:00
moneromooo-monero
d753d716a6
fix a few leaks by throwing objects, not newed pointers to objects 2017-12-18 15:15:13 +00:00
moneromooo-monero
09ce03d612
move includes around to lessen overall load 2017-12-16 22:46:38 +00:00
flozilla
f9fad18608
blockchain_db: sanity check on tx/hash vector sizes
It could trip on a corrupt/crafted file if the user has disabled
input verification.
2017-11-15 10:02:23 +00:00
Riccardo Spagni
5de492be40
Merge pull request #2629
383ff4f6 remove "using namespace std" from headers (moneromooo-monero)
2017-11-14 21:29:49 +02:00
moneromooo-monero
383ff4f689
remove "using namespace std" from headers
It's nasty, and actually breaks on Solaris, where if.h fails to
build due to:

  struct map *if_memmap;
2017-11-14 16:56:10 +00:00
Riccardo Spagni
0d0ce8c21e
Merge pull request #2742
7c7d3672 Increase LMDB maxreaders if large number of threads in use (Howard Chu)
6738753b Use max_concurrency as-is (Howard Chu)
2017-11-14 15:30:03 +02:00
Riccardo Spagni
fd0740e5e7
Merge pull request #2615
10013e94 Protect node privacy by proper filtering in restricted-mode RPC answers (binaryFate)
2017-11-14 14:53:10 +02:00
binaryFate
10013e9434 Protect node privacy by proper filtering in restricted-mode RPC answers
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode.
This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected.
In practice, when running with `--restricted-rpc`:
* get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero.
* get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions
* get_transaction_pool_hashes.bin will not list such transaction
* get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed

The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality.
Fixes #2590.
2017-11-08 13:07:07 +01:00
moneromooo-monero
ccf53a566c
track double spending in the txpool
Transactions in the txpool are marked when another transaction
is seen double spending one or more of its inputs.
This is then exposed wherever appropriate.

Note that being marked with this "double spend seen" flag does
NOT mean this transaction IS a double spend and will never be
mined: it just means that the network has seen at least another
transaction spending at least one of the same inputs, so care
should be taken to wait for a few confirmations before acting
upon that transaction (ie, mostly of use for merchants wanting
to accept unconfirmed transactions).
2017-11-06 00:05:44 +00:00
Howard Chu
7c7d36725a
Increase LMDB maxreaders if large number of threads in use 2017-11-02 16:33:48 +00:00
ston1th
f946bb7be6 fixed an LMDB issue on OpenBSD (#2699) which leads to a crash in monerod 2017-10-26 17:16:06 +02:00
Riccardo Spagni
b293ad9a1a
Merge pull request #2640
22b51e06 db_lmdb: include chain height when failing to find an output key (moneromooo-monero)
5db433b3 blockchain: avoid exceptions in output verification (moneromooo-monero)
2017-10-15 18:59:45 +02:00
moneromooo-monero
22b51e0672
db_lmdb: include chain height when failing to find an output key
To help debugging logs.
2017-10-12 13:28:28 +01:00
moneromooo-monero
bc1b93333e
frob level 1 logs a bit for consistency
Level 1 logs map to INFO, so setting log level to 1 should
show these. Demote some stuff to DEBUG to avoid spam, though.
2017-10-07 20:03:42 +01:00
Riccardo Spagni
e5a2cdbfad
Merge pull request #2457
d2d8a41c Use actual batch size for resize estimates (Howard Chu)
2017-09-25 16:57:07 +02:00
moneromooo-monero
9aeae7a224
db_lmdb: fix use of uninitialized key in for_blocks_range 2017-09-19 14:11:24 +01:00
Howard Chu
d2d8a41c47
Use actual batch size for resize estimates
And optimize import startup:

Remember start_height position during initial count_blocks pass
to avoid having to reread entire file again to arrive at start_height
2017-09-17 22:08:40 +01:00
Riccardo Spagni
1e57e48342
Merge pull request #2372
c867357a cryptonote_protocol: error handling on cleanup_handle_incoming_blocks (moneromooo-monero)
ce901fcb Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocks (moneromooo-monero)
84fa015e core: guard against exceptions in handle_incoming_{block,tx} (moneromooo-monero)
2017-09-02 11:30:57 +02:00
moneromooo-monero
ce901fcb31
Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocks 2017-08-29 15:43:32 +01:00
moneromooo-monero
a95e460c71
move db specific options to BlockchainDB
Avoids common depending on blockchain_db, which can cause
link errors.
2017-08-29 11:43:34 +01:00
Riccardo Spagni
36b978f50e
Merge pull request #2314
c22d22e2 Cleanup test impact of adding safesyncmode() method (Howard Chu)
9a859844 Toggle SAFE syncmode on and off automatically (Howard Chu)
2017-08-26 23:43:22 +02:00
Howard Chu
9a859844f4
Toggle SAFE syncmode on and off automatically
If monerod is started with default sync mode, set it to SAFE after
synchronization completes. Set it back to FAST if synchronization
restarts (e.g. because another peer has a longer blockchain).

If monerod is started with an explicit sync mode, none of this
automation takes effect.
2017-08-20 16:30:28 +01:00
Howard Chu
80344740bd
More DB support cleanup
Hide DB types from db_types.h - no reason to recompile dependencies
when DB types change.

Also remove lingering in-memory DB references, they've been
obsolete since 9e82b694da
2017-08-20 13:57:36 +01:00
Howard Chu
4c7f8ac04f
DB cleanup
Hide LMDB-specific stuff behind blockchain_db.h. Nobody besides blockchain_db.cpp
should ever be including DB-specific headers any more.
2017-08-19 18:11:38 +01:00
moneromooo-monero
c6e200a8ab
core: add --db-salvage command line flag
Use to load the database when the primary meta page is corrupted
2017-08-12 19:04:44 +01:00
moneromooo-monero
214fd81e93
some include cleanup 2017-07-31 16:36:52 +01:00
moneromooo-monero
235df7f484
blockchain_db: add a txpool tx getter which returns existence
Avoids exception spam for the "nope, not found" case
2017-06-11 15:36:48 +01:00
Howard Chu
d17c0fc2d0
Don't copy blockchain for coinbase_tx_sum
Changed Blockchain::for_all_blocks() to for_blocks_range()
Operate on blockchain in-place instead of building a copy first.
2017-06-01 14:14:24 +01:00
Riccardo Spagni
a4c13ea092
Merge pull request #2039
4b932ff3 changed crypto to cncrypto so it generated libcncrypto (Gentian)
2017-05-30 21:24:11 +02:00
moneromooo-monero
b52abd1370
Move txpool to the database
Integration could go further (ie, return_tx_to_pool calls should
not be needed anymore, possibly other things).

poolstate.bin is now obsolete.
2017-05-25 22:23:37 +01:00
Gentian
4b932ff314 changed crypto to cncrypto so it generated libcncrypto
fix a cmakelist
2017-05-23 07:45:40 -04:00
Howard Chu
d3aaf74080
Partial fix #1876, check thread in block_txn_start()
Don't allow use of existing batch txn if it's from the wrong thread
2017-03-19 12:27:03 +00:00
kenshi84
7d07c64fe5
fix dependency: put HardFork back to cryptonote_basic, made some BlockchainDB functions virtual again to avoid missing symbols error 2017-03-10 11:22:39 +09:00
Howard Chu
eb62dcc871
Remove redundant num_txs() method
Cleanup of bf1348b7e2
2017-03-03 16:52:45 +00:00
moneromooo-monero
12adb4a3f3
core: move hardfork back to cryptonote_core
should fix a cross dependency betewen cryptonote_basic and
blockchain_db
2017-02-25 16:41:35 +00:00
Riccardo Spagni
c3599fa7b9
update copyright year, fix occasional lack of newline at line end 2017-02-21 19:38:18 +02:00
Riccardo Spagni
49efd3add9
Merge pull request #1727
0288310e blockchain_db: add "raw" blobdata getters for block and transaction (moneromooo-monero)
2017-02-21 11:27:15 +02:00
Riccardo Spagni
59d96eedd8
Merge pull request #1724
cca95c1c blockchain_db: do not throw on expected partial results getting keys (moneromooo-monero)
2017-02-21 11:23:03 +02:00
Howard Chu
2e9136767e
Handle map resizes from other processes 2017-02-18 22:22:24 +00:00
Howard Chu
bf1348b7e2
Can't cache num_txs or num_outputs either
Same reason as 3ff54bdd7a
2017-02-18 21:01:24 +00:00
Howard Chu
dc53e9eef2
Add a few read txns to streamline
Slight perf gain, but mainly to reduce spam at loglevel 3
2017-02-18 21:01:24 +00:00