Commit Graph

123 Commits

Author SHA1 Message Date
mj-xmr da9aa1f7f8
Copyright: Update to 2022 2022-03-04 06:59:20 +01:00
Howard Chu 5f54dea4ec
LMDB: fix deadlock in resized detection 2021-09-22 15:47:04 +01:00
moneromooo-monero 1eb14af1a3
rpc: limit the number of txes for get_blocks.bin 2020-12-29 04:31:58 +00:00
moneromooo-monero 55363c5941
Avoid some temporary strings when reading off the database 2020-08-17 14:02:27 +00:00
Alexander Blair 6d6c691a0f
Merge pull request #6536
bd9653663 db_lmdb: test for mmap support at init time (moneromooo-monero)
2020-07-19 03:37:15 -07:00
Alexander Blair 36d50d93f2
Merge pull request #6534
7bd66b01b daemon: guard against rare 'difficulty drift' bug with checkpoints and recalculation (stoffu)
2020-07-19 03:36:39 -07:00
stoffu 7bd66b01bf
daemon: guard against rare 'difficulty drift' bug with checkpoints and recalculation
On startup, it checks against the difficulty checkpoints, and if any mismatch is found, recalculates all the blocks with wrong difficulties. Additionally, once a week it recalculates difficulties of blocks after the last difficulty checkpoint.
2020-06-09 10:40:51 +09:00
moneromooo-monero bd96536637
db_lmdb: test for mmap support at init time
It'll make it clearer when a DB init failure is due to being
on a filesystem which does not support mmap
2020-05-16 00:20:22 +00:00
SomaticFanatic 5ef0607da6 Update copyright year to 2020
Update copyright year to 2020
2020-05-06 22:36:54 -04:00
moneromooo-monero 8958b4e7aa
blockchain_db: faster fetching of consecutive txes
Useful for wallet refresh or node sync
2020-02-27 15:05:34 +00:00
Alexander Blair 8eedc8a390
Merge pull request #6140
08635a08 blockchain: speedup fetching pruned contiguous tx blobs (moneromooo-monero)
2020-01-25 16:55:18 -08:00
moneromooo-monero 08635a0875
blockchain: speedup fetching pruned contiguous tx blobs
About twice as fast, very roughly
2019-11-15 13:23:30 +00:00
Lee Clagett 5d7ae2d279 Adding support for hidden (anonymity) txpool 2019-11-02 20:36:03 +00:00
moneromooo-monero 06b8f29992
blockchain: keep alternative blocks in LMDB
Alternative blocks are cleared on startup unless --keep-alt-blocks
is passed on the command line
2019-05-08 17:36:52 +00:00
moneromooo-monero 5e673c03fe
blockchain_db: fix db txn ending too early
The db txn in add_block ending caused the entire overarching
batch txn to stop.
Also add a new guard class so a db txn can be stopped in the
face of exceptions.

Also use a read only db txn in init when the db itself is
read only, and do not save the max tx size in that case.
2019-04-14 08:35:38 +00:00
moneromooo-monero 089c7637a6
cryptonote: rework block blob size sanity check
Use the actual block weight limit, assuming that weight is always
greater or equal to size
2019-04-05 09:35:19 +00:00
moneromooo-monero 91f4c7f45f
Make difficulty 128 bit instead of 64 bit
Based on Boolberry work by:
  jahrsg <jahr@jahr.me>
  cr.zoidberg <crypto.zoidberg@gmail.com>
2019-03-24 21:03:19 +00:00
Riccardo Spagni 5ac46c5310
Merge pull request #5256
4b21d38d blockchain: speed up getting N blocks weights/long term weights (moneromooo-monero)
2019-03-21 14:48:40 +02:00
moneromooo-monero 4b21d38dfd
blockchain: speed up getting N blocks weights/long term weights 2019-03-08 12:04:14 +00:00
binaryFate 1f2930ce0b Update 2019 copyright 2019-03-05 22:05:34 +01:00
moneromooo-monero 79b4e9f377
save some database calls when getting top block hash and height 2019-03-05 11:58:05 +00:00
moneromooo-monero b044d03a51
Avoid repeated (de)serialization when syncing 2019-03-05 11:57:55 +00:00
moneromooo-monero b8787f4302
ArticMine's new block weight algorithm
This curbs runaway growth while still allowing substantial
spikes in block weight

Original specification from ArticMine:

here is the scaling proposal
Define: LongTermBlockWeight
Before fork:
LongTermBlockWeight = BlockWeight
At or after fork:
LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight)
Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time.
Define:   LongTermEffectiveMedianBlockWeight
LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight))
Change Definition of EffectiveMedianBlockWeight
From (current definition)
EffectiveMedianBlockWeight  = max(300000, MedianOverPrevious100Blocks(BlockWeight))
To (proposed definition)
EffectiveMedianBlockWeight  = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight)
Notes:
1) There are no other changes to the existing penalty formula, median calculation, fees etc.
2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This  is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork.
3) When the  EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty.

Note: the long term block weight is stored in the database, but not in the actual block itself,
since it requires recalculating anyway for verification.
2019-03-04 09:33:58 +00:00
moneromooo-monero b750fb27b0
Pruning
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.

No other data is currently pruned.

There are three ways to prune a blockchain:

- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility

The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.

The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.

Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
2019-01-22 20:30:51 +00:00
moneromooo-monero a6216d1ac2
blockchain_db: allow getting output keys without commitment
Since the commitment has to be calculated for non rct outputs,
it slows down a lot unnecessarily if we don't need it
2019-01-16 20:13:08 +00:00
Riccardo Spagni 9d57ee9ff6
Merge pull request #4984
008647d7 blockchain_db: speedup tx output gathering (moneromooo-monero)
2019-01-16 19:06:07 +02:00
moneroexamples ec68cf822f Make get_output_key method const
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs.

Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
2019-01-07 05:53:14 +08:00
luigi1111 69e8567c0e
Merge pull request #4946
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero)
ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero)
8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero)
5511563 db_lmdb: avoid pointless division (moneromooo-monero)
d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero)
9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
2018-12-31 16:06:50 -06:00
moneromooo-monero 008647d7eb
blockchain_db: speedup tx output gathering
We know all the data we'll want for getblocks.bin is contiguous
2018-12-18 11:55:21 +00:00
moneromooo-monero 833269834d
db_lmdb: inline check_open, it's trivial and called everywhere 2018-12-05 21:13:21 +00:00
moneromooo-monero 6644b9b7b7
blockchain_db: remove a couple unused functions 2018-12-05 21:13:13 +00:00
moneromooo-monero 17b45725af
Outputs where all amounts are known spent can now be pruned
Only for pre rct for obvious reasons.

Note: DO NOT use a known spent list which includes outputs
which are not known spent. If the list includes any output
that's just strongly thought to be spent, but not provably
so, you risk finding yourself unable to sync past the point
where that output is spent.

I estimate only 200 MB saved on current mainnet though,
unless the new blackballing rule unearths a good amount of
large-amount-set extra spent outs.
2018-11-27 15:45:42 +00:00
moneromooo-monero fc98f7a0a1
rpc: speedup get_outs.bin 2018-11-26 18:56:23 +00:00
moneromooo-monero 6ecc99ad1f
core: avoid unnecessary tx/blob conversions 2018-11-01 15:51:19 +00:00
moneromooo-monero 5ffb2ff9b7
v8: per byte fee, pad bulletproofs, fixed 11 ring size 2018-09-11 13:38:07 +00:00
luigi1111 46bcd91db4
Merge pull request #4256
d6fc870 db_lmdb: resize blockchain database when 90% filled (moneromooo-monero)
2018-09-10 15:04:50 -05:00
moneromooo-monero d6fc870d04
db_lmdb: resize blockchain database when 90% filled
instead of a random ratio from 60% to 90%.
2018-08-14 08:55:49 +00:00
moneromooo-monero 43f71100bc
blockchain_db: remove unused get_output_key variant
It was actually incorrect, as it would not return commitment
2018-07-30 21:57:47 +00:00
luigi1111 3e026ff6ed
Merge pull request #4013
e5592c4 rpc: add blockchain disk size to getinfo (moneromooo-monero)
2018-07-19 13:40:42 -05:00
luigi1111 21afa71ba6
Merge pull request #3981
45e419b db: store cumulative rct output distribution in the db for speed (moneromooo-monero)
2018-07-19 13:39:24 -05:00
moneromooo-monero 45e419bd5c
db: store cumulative rct output distribution in the db for speed
This gets rid of the temporary precalc cache.

Also make the RPC able to send data back in binary or JSON,
since there can be a lot of data

This bumps the LMDB database format to v3, with migration.
2018-07-13 11:37:04 +01:00
victorsintnicolaas 34cb6b4b70 add --regtest and --fixed-difficulty for regression testing
on_generateblocks RPC call combines functionality from the on_getblocktemplate and on_submitblock RPC calls to allow rapid block creation. Difficulty is set permanently to 1 for regtest.
Makes use of FAKECHAIN network type, but takes hard fork heights from mainchain
Default reserve_size in generate_blocks RPC call is now 1. If it is 0, the following error occurs 'Failed to calculate offset for'.
Queries hard fork heights info of other network types
2018-06-29 10:06:11 +02:00
moneromooo-monero e5592c4bab
rpc: add blockchain disk size to getinfo
This should help new nodes predict how much disk space will be
needed for a full sync
2018-06-20 23:48:37 +01:00
stoffu 149da420e9 db_lmdb: enable batch transactions by default 2018-06-14 20:56:11 +02:00
moneromooo-monero b9389e582e
db_lmdb: save pruned and prunable tx data separately
This bumps DB version to 2, migration code will run for v1 DBs
2018-05-23 22:48:12 +01:00
moneromooo-monero 5f8ffca34d
speed up get_output_distribution (and precalc common case) 2018-05-21 19:51:58 +01:00
moneromooo-monero ffeeefde60
speedup get_output_histogram for all amounts when min_count > 0
This skips the vast majority of "dust" output amounts with just
one instance on the chain. Clocks in at 0.15% of the original
time on testnet.
2018-04-23 10:54:28 +01:00
moneromooo-monero 41f727ce42
add RPC to get a histogram of outputs of a given amount 2018-03-16 10:32:26 +00:00
Riccardo Spagni 260d666cf4
Merge pull request #3231
84a8b2da Don't create readtxn until after txn_safe gate check (Howard Chu)
2018-02-16 14:24:01 +01:00
Howard Chu 84a8b2da14
Don't create readtxn until after txn_safe gate check 2018-01-31 18:18:01 +00:00