Commit graph

92 commits

Author SHA1 Message Date
Riccardo Spagni
9d57ee9ff6
Merge pull request #4984
008647d7 blockchain_db: speedup tx output gathering (moneromooo-monero)
2019-01-16 19:06:07 +02:00
moneroexamples
ec68cf822f Make get_output_key method const
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs.

Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
2019-01-07 05:53:14 +08:00
moneromooo-monero
008647d7eb
blockchain_db: speedup tx output gathering
We know all the data we'll want for getblocks.bin is contiguous
2018-12-18 11:55:21 +00:00
Riccardo Spagni
d855f9bb92
Merge pull request #4901
a48f2dab blockchain_prune_known_spent_data: blackball file is now optional (moneromooo-monero)
17b45725 Outputs where all amounts are known spent can now be pruned (moneromooo-monero)
2018-12-12 11:57:31 +02:00
Riccardo Spagni
c00ac446fd
Merge pull request #4854
bd98e99c Removed a lot of unnecessary includes (Martijn Otto)
2018-12-04 17:08:42 +02:00
moneromooo-monero
17b45725af
Outputs where all amounts are known spent can now be pruned
Only for pre rct for obvious reasons.

Note: DO NOT use a known spent list which includes outputs
which are not known spent. If the list includes any output
that's just strongly thought to be spent, but not provably
so, you risk finding yourself unable to sync past the point
where that output is spent.

I estimate only 200 MB saved on current mainnet though,
unless the new blackballing rule unearths a good amount of
large-amount-set extra spent outs.
2018-11-27 15:45:42 +00:00
moneromooo-monero
fc98f7a0a1
rpc: speedup get_outs.bin 2018-11-26 18:56:23 +00:00
Martijn Otto
bd98e99c80
Removed a lot of unnecessary includes 2018-11-15 17:29:34 +01:00
moneromooo-monero
6ecc99ad1f
core: avoid unnecessary tx/blob conversions 2018-11-01 15:51:19 +00:00
moneromooo-monero
4eca42b241
blockchain_db: initialize m_hardfork in ctor just in case
Coverity 136568
2018-10-16 11:58:46 +00:00
moneromooo-monero
5ffb2ff9b7
v8: per byte fee, pad bulletproofs, fixed 11 ring size 2018-09-11 13:38:07 +00:00
moneromooo-monero
43f71100bc
blockchain_db: remove unused get_output_key variant
It was actually incorrect, as it would not return commitment
2018-07-30 21:57:47 +00:00
luigi1111
3e026ff6ed
Merge pull request #4013
e5592c4 rpc: add blockchain disk size to getinfo (moneromooo-monero)
2018-07-19 13:40:42 -05:00
luigi1111
21afa71ba6
Merge pull request #3981
45e419b db: store cumulative rct output distribution in the db for speed (moneromooo-monero)
2018-07-19 13:39:24 -05:00
luigi1111
025187e6c9
Merge pull request #3854
149da42 db_lmdb: enable batch transactions by default (stoffu)
34cb6b4 add --regtest and --fixed-difficulty for regression testing (vicsn)
9e1403e update get_info RPC and bump RPC version (vicsn)
207b66e first new functional tests (vicsn)
2018-07-19 13:35:25 -05:00
moneromooo-monero
45e419bd5c
db: store cumulative rct output distribution in the db for speed
This gets rid of the temporary precalc cache.

Also make the RPC able to send data back in binary or JSON,
since there can be a lot of data

This bumps the LMDB database format to v3, with migration.
2018-07-13 11:37:04 +01:00
victorsintnicolaas
34cb6b4b70 add --regtest and --fixed-difficulty for regression testing
on_generateblocks RPC call combines functionality from the on_getblocktemplate and on_submitblock RPC calls to allow rapid block creation. Difficulty is set permanently to 1 for regtest.
Makes use of FAKECHAIN network type, but takes hard fork heights from mainchain
Default reserve_size in generate_blocks RPC call is now 1. If it is 0, the following error occurs 'Failed to calculate offset for'.
Queries hard fork heights info of other network types
2018-06-29 10:06:11 +02:00
moneromooo-monero
e5592c4bab
rpc: add blockchain disk size to getinfo
This should help new nodes predict how much disk space will be
needed for a full sync
2018-06-20 23:48:37 +01:00
moneromooo-monero
d81e042306
tx_pool: initialize bitflags padding since it gets written to storage
Avoids valgrind reporting uninitialized data usage
2018-06-05 20:26:44 +01:00
moneromooo-monero
b9389e582e
db_lmdb: save pruned and prunable tx data separately
This bumps DB version to 2, migration code will run for v1 DBs
2018-05-23 22:48:12 +01:00
moneromooo-monero
5f8ffca34d
speed up get_output_distribution (and precalc common case) 2018-05-21 19:51:58 +01:00
moneromooo-monero
ffeeefde60
speedup get_output_histogram for all amounts when min_count > 0
This skips the vast majority of "dust" output amounts with just
one instance on the chain. Clocks in at 0.15% of the original
time on testnet.
2018-04-23 10:54:28 +01:00
moneromooo-monero
41f727ce42
add RPC to get a histogram of outputs of a given amount 2018-03-16 10:32:26 +00:00
moneromooo-monero
ad11db9144
blockchain_db: initialize m_open in base class ctor
It's cleaner this way, since it's a base class field

Coverity 136568
2018-02-02 15:21:39 +00:00
xmr-eric
18216f19dd Update 2018 copyright 2018-01-26 10:03:20 -05:00
moneromooo-monero
ae8602303a
Fix exceptions not finding txpool txes when relaying 2017-12-20 13:02:53 +00:00
moneromooo-monero
09ce03d612
move includes around to lessen overall load 2017-12-16 22:46:38 +00:00
Riccardo Spagni
fd0740e5e7
Merge pull request #2615
10013e94 Protect node privacy by proper filtering in restricted-mode RPC answers (binaryFate)
2017-11-14 14:53:10 +02:00
binaryFate
10013e9434 Protect node privacy by proper filtering in restricted-mode RPC answers
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode.
This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected.
In practice, when running with `--restricted-rpc`:
* get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero.
* get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions
* get_transaction_pool_hashes.bin will not list such transaction
* get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed

The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality.
Fixes #2590.
2017-11-08 13:07:07 +01:00
moneromooo-monero
ccf53a566c
track double spending in the txpool
Transactions in the txpool are marked when another transaction
is seen double spending one or more of its inputs.
This is then exposed wherever appropriate.

Note that being marked with this "double spend seen" flag does
NOT mean this transaction IS a double spend and will never be
mined: it just means that the network has seen at least another
transaction spending at least one of the same inputs, so care
should be taken to wait for a few confirmations before acting
upon that transaction (ie, mostly of use for merchants wanting
to accept unconfirmed transactions).
2017-11-06 00:05:44 +00:00
Howard Chu
d2d8a41c47
Use actual batch size for resize estimates
And optimize import startup:

Remember start_height position during initial count_blocks pass
to avoid having to reread entire file again to arrive at start_height
2017-09-17 22:08:40 +01:00
moneromooo-monero
a95e460c71
move db specific options to BlockchainDB
Avoids common depending on blockchain_db, which can cause
link errors.
2017-08-29 11:43:34 +01:00
Howard Chu
9a859844f4
Toggle SAFE syncmode on and off automatically
If monerod is started with default sync mode, set it to SAFE after
synchronization completes. Set it back to FAST if synchronization
restarts (e.g. because another peer has a longer blockchain).

If monerod is started with an explicit sync mode, none of this
automation takes effect.
2017-08-20 16:30:28 +01:00
Howard Chu
4c7f8ac04f
DB cleanup
Hide LMDB-specific stuff behind blockchain_db.h. Nobody besides blockchain_db.cpp
should ever be including DB-specific headers any more.
2017-08-19 18:11:38 +01:00
moneromooo-monero
235df7f484
blockchain_db: add a txpool tx getter which returns existence
Avoids exception spam for the "nope, not found" case
2017-06-11 15:36:48 +01:00
Howard Chu
d17c0fc2d0
Don't copy blockchain for coinbase_tx_sum
Changed Blockchain::for_all_blocks() to for_blocks_range()
Operate on blockchain in-place instead of building a copy first.
2017-06-01 14:14:24 +01:00
moneromooo-monero
b52abd1370
Move txpool to the database
Integration could go further (ie, return_tx_to_pool calls should
not be needed anymore, possibly other things).

poolstate.bin is now obsolete.
2017-05-25 22:23:37 +01:00
kenshi84
7d07c64fe5
fix dependency: put HardFork back to cryptonote_basic, made some BlockchainDB functions virtual again to avoid missing symbols error 2017-03-10 11:22:39 +09:00
moneromooo-monero
12adb4a3f3
core: move hardfork back to cryptonote_core
should fix a cross dependency betewen cryptonote_basic and
blockchain_db
2017-02-25 16:41:35 +00:00
Riccardo Spagni
c3599fa7b9
update copyright year, fix occasional lack of newline at line end 2017-02-21 19:38:18 +02:00
Riccardo Spagni
49efd3add9
Merge pull request #1727
0288310e blockchain_db: add "raw" blobdata getters for block and transaction (moneromooo-monero)
2017-02-21 11:27:15 +02:00
moneromooo-monero
0288310e3b
blockchain_db: add "raw" blobdata getters for block and transaction
This speeds up operations such as serving blocks to syncing peers
2017-02-13 21:11:37 +00:00
moneromooo-monero
cca95c1c7a
blockchain_db: do not throw on expected partial results getting keys
When scanning for outputs used in a set of incoming blocks,
we expect that some of the inputs in their transactions will
not be found in the blockchain, as they could be in previous
blocks in that set. Those outputs will be scanned there at
a later point. In this case, we add a flag to control wehther
an output not being found is expected or not.
2017-02-13 19:05:30 +00:00
kenshi84
8027ce0c75 extract some basic code from libcryptonote_core into libcryptonote_basic 2017-02-08 22:45:15 +09:00
Riccardo Spagni
65e33b1bc5
Merge pull request #1506
3ff54bdd Check for correct thread before ending batch transaction (Howard Chu)
eaf8470b Must wait for previous batch to finish before starting new one (Howard Chu)
c903c554 Don't cache block height, always get from DB (Howard Chu)
eb1fb601 Tweak default db-sync-mode to fast:async:1 (Howard Chu)
0693cff9 Use batch transactions when syncing (Howard Chu)
2017-01-15 14:43:12 -05:00
Howard Chu
0693cff925
Use batch transactions when syncing
Faster throughput while avoiding corruption. I.e., makes
running with --db-sync-mode safe more tolerable.
2017-01-14 22:43:06 +00:00
moneromooo-monero
0478ac6848
blockchain: allow marking "tx not found" without an exception
This is a normal occurence in many cases, and there is no need
to spam the log with those when it is.
2017-01-07 20:52:17 +00:00
moneromooo-monero
88faec75fe
wallet: select part of the fake outs from recent outputs
25% of the outputs are selected from the last 5 days (if possible),
in order to avoid the common case of sending recently received
outputs again. 25% and 5 days are subject to review later, since
it's just a wallet level change.
2016-10-15 18:17:16 +01:00
moneromooo-monero
6cf8ca2a7f
core: faster find_blockchain_supplement
Since this queries block heights for blocks that may or may not
exist, queries for non existing blocks would throw an exception,
and that would slow down the loop a lot. 7 seconds to go through
a 30 hash list.

Fix this by adding an optional return block height to block_exists
and using this instead. Actual errors will still throw an
exception.

This also cuts down on log exception spam.
2016-08-31 10:03:32 +01:00
moneromooo-monero
59a66e209a
move the rct commitments to the output_amounts database
Since these are needed at the same time as the output pubkeys,
this is a whole lot faster, and takes less space. Only outputs
of 0 amount store the commitment. When reading other outputs,
a fake commitment is regenerated on the fly. This avoids having
to rewrite the database to add space for fake commitments for
existing outputs.

This code relies on two things:

- LMDB must support fixed size records per key, rather than
per database (ie, all records on key 0 are the same size, all
records for non 0 keys are same size, but records from key 0
and non 0 keys do have different sizes).

- the commitment must be directly after the rest of the data
in outkey and output_data_t.
2016-08-28 21:29:02 +01:00