Bug fixes, golangci-lint update version #754
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add spine blockquery to blocksmithstrategyspine to be able to get the previous spine block when selecting/sorting blocksmiths
* get escrow transactions with filter fields and pagination * get escrow transactions functional test
* #592 add fee model to action type * #592 add fee model to action type * #592 ignore package-interface naming lint rule * #592 update pb file of transaction : add model + service for getTxMinimumFee api * #592 add handler for calculate minimum fee api * #592 fix GetTransactionMinimumFee request * #592 fix wrong type * #592 add api client for testing get minimum fee
* Logic for snapshot file creation * Add new interface and service to abstract snapshot file creation for different block types (for now only mainchain) * Add ugorji/go/codec package to add CBOR serialization support * Added Logic for snapshot file processing (create/download) * Add new interface and service to abstract snapshot file download * Implemented some of the logic in snapshotMainBlockService for downloading and creating new snapshot files/chunks * Added business logic for snapshots * Add new service FileService to abstract (and make testable) methods relative to file manipulation * Better separation of logic between shapshotservice (generic methods) and snapshotBlockService (blocktype-specific methods) * Move GetFileNameFromHash and GetHashFromFileName into fileservice * Add query and service handler to retrieve account balances for snapshots * Add minimal process management for snapshot files: if a snapshot of the same block type is still running and a new one is started, the new one will stop the old one (this logic must be discussed and is a WIP) * Add initial logic to create a snapshot file containing all (only last state) account balances * TODO: increase code coverage (few methods are tested so far) * Bug fix * Add EscrowTransactionQuery to GetDerivedQueries * Added logic for generating snapshot files * Add All relevant queries and wrappers (to ease testing) to snapshotMainBlockService * Add test case for NewSnapshotFile and new queries * Finalised logic for generating snapshot files (first iteration only has single file per snapshot) * Add all relevant queries to get snapshot's payload * Add method to parse and import snapshot into db * Add config parameter for snapshots folder * Add unit and integration tests * Added logic for download snapshots * Add method to parse the spineBlockManifest and download file chunk hashes * TODO: download file is still mocked * Uncomment logic previously commented for testing purposes * Increase MainchainSnapshotInterval value to 1440 to avoid having too many snapshots in dev and production environments * Add query interface for snapshot-specific queries (SnapshotQuery) * Add SelectDataForSnapshot interface signature to new interface * Add GetSnapshotQuery method to get all snapshot queries implementations (in a map) * Refactor relative query methods in every relevant repository * Refactor some constant to use time.Duration instead of int type * Fix unit tests * Add DESC to order by clauses in snapshot queries * Refactor GetChainTypeCount to GetChainTypes to make chain types countable * Remove temporary test function * Use SnapshotQuery interface to loop through all snapshot query implementations instead of adding them one by one (less code duplication) * Use SnapshotQuery interface to loop through all snapshot query implementations instead of adding them one by one (less code duplication)
* #596 add multisignature transaction body * #596 add constants for MultiSignature body fields * #596 add multisignature tx type skeleton * #596 add multisig tx type enum * #596 add command tools for generate multisignature tx and parsing any tx * #596 basic multisignature parsing functionality * #596 update schema commit * #596 add signature info * #596 update signature structure to use map * #596 adjust cmd tools to new multisig byte structure * #596 update multisig transaction bytes * #596 update schema
* Added proto message for snapshots * Add SnapshotPayload message representing internal snapshot data structure * Added logic for snapshot chunk 'basic' strategy * Add Strategy (interface and implementation) for 'basic' snapshot chunks: chunks are generated by splitting a full encoded snapshot * Add SnapshotPayload model to schema, since we are using it in two different packages * Refactor snapshotmainblockservice to use new snapshot chunk strategy * Refactor unit and integration tests * Completing logic for snapshot basic chunk strategy * Finalize BuildSnapshotFromChunks and its test * Add ReadFileByHash to FileService to be able to directly read a file from its hash-name and mock ioutil.ReadFile in services that depend on this service * Remove unused parameter 'chunkSizeBytes' from SnapshotBlockServiceInterface.NewSnapshotFile
* #600 add new managed table for multisig * #600 update multisig managed table model * #600 add managed table query for multisig Co-authored-by: Ariasa Ketut <[email protected]>
* method multisig account address assembler * command line tool multisig generator command * set 1.20.0 of golangci-lint version to fixed circleci fail
…ad (#608) * Refactor spineBlockDownloadService to blockStatusService, to be able to set/get all chaintype blockchain statuses (for now only if a node has finished downloading the block first time after has started) * Refactor downloadBlockchain to use blockStatusService as dependency and set blockchain download status referenced by blockStatusService * Simplify main.go by moving logic to set first blockchain download into downloadBlockchain service * Add new blocker type P2PNetworkConnectionErr * Increase timers for checking block download process * Refactor blockStatusService and implementations to blockTypeStatusService * Refactor relative code in other services * Remove cast to int snapshotPayloadHeight in snapshotMainBlockService * Fix bug in snapshotPayloadHeight computation * Refactor constructor method for BlockchainSyncService
* no need to validate current block height * fixed transcation generator afftected by fee
* Fix logic for unlocking smithing process based on network status and blockchain download status (all main blocks downloaded) * Add new service variable and relative getter/setter to blockTypeStatusService 'isSmithingLocked' to control whether smithing is locked or not, regardless of the blockchain download status * Remove unused getter from chainType * Add new constants for better code readability * Fix tests
* no need to validate current block height * set index into some tables for speed up latency get a lot of records * remove unused a line of codes
* no need to validate current block height * functionality test get escrow by id
* Change fatalf to errorf/return in goroutine in snapshotService.go * Add missing return statement for error in snapshotService.go * Added logic for downloading files from a random peer and Code Refactoring * Refactor filedownloader: moved to p2p package because being in service generates a cycle dependecy * Add rpc call (client request and server response) to download a file from a random peer * Add logic to download/parse/verify a file and a snapshot * Added logic and file download metrics * Add logic to download a file, server side * Add metrics to count succeeded/failed snapshot downloads * Update schema: add Failed files for file download * Add metric IncrementSnapshotDownloadCounter to count number of downloaded files from every node * Refactor InsertSpineBlockManifest to not fail if a new record with same id is inserted, to allow downloaded spine block manifests to override locally (probably wrong) generated ones * Rename megablockQuery to spineBlockManifestQuery * Refactor blockchainStatusService to use safer (for r/w race conditions) and faster sync.Map instead of regular map + sync.mutex * Fix bug in spineBlockService.push: insert spineBlockManifest records if they are present as part of block data * Fix smith process lock when downloading spine and main blocks * Fix download blockchain process: spine blocks, snapshot, rest of main blocks. then start smithing * Add retry download failed file chunks from different peers * WIP: commented code to make it easier for others to run test nodes locally and double check the download process works as expected. All commented code is prepend with "// STEF" string * Simplified fileService GetFileNameFromHash and GetFileNameFromBytes so that they don't have to return error and refactored relative code that uses them * Add basic unit test for p2p.DownloadFilesFromPeer (still needs more test cases) * Optimize p2p.DownloadFilesFromPeer while writing the test * Fix code after code review * fixed bug in caclulating snapshot's height * Add two struct and methods to mimic sync.Map behaviour, but with regular map + sync.RWMutex (more efficient on machines with low cpu core count) * Refactor code that used sync.Map in favor of new map struct * Add util function to calculate a quick (non secure) random number * Fix uncaught nil in p2pServerHandler * Add NewSpineBlockManifestQuery to derivedqueries to make it rollback safe (it must be rolledback together with mainchain blocks) * Add a couple of TODOs for spine block manifest validation * Fix bug in unitialized model.MapIntBool struct * Add missing test for GenerateSnapshot method * Uncommented code for testing * Uncomment code * Fix one unit test after fixing the snapshot height (previous commit) Co-authored-by: stefano galassi <[email protected]>
* add bitcoin sig, btcd & btcutil package * restructure Ed25519 signature type function * add cmd to generate bitcoin account * add signature-type flag in cmd tx generator * cmd enhace switch case generate account * return error when sign and verify payload * cmd generate tx with sender signature type * adjust general parse transaction bytes * adjust unit test * udate cmd readme * cmd, panic when invalid signature type * remove unused function * add generate account in signature * add signature unit test * add misisng import package * update schema * change parameter type of GenerateAccountFromSeed Co-authored-by: Ariasa Ketut <[email protected]>
* no need to validate current block height * update circleci config use go modules instead of dep and update go version to 1.14 * go mod and more cleaning shadow declaration variable * Revert "GRPC endpoint get single escrow transaction by ID (#620)" This reverts commit 7824924. * Revert "Revert "GRPC endpoint get single escrow transaction by ID (#620)"" This reverts commit bedbf87. * revert unused commit * remove Gopkg.* files * remove no needed comment * make sure go modules cache correctly * change circleci working directory, update readme * update path of cache * fixed go modules caches
* #603 initialize multisig service layer * #603 implement apply confirmed * #603 add sender address in pending transaction * #603 update multisignature.PendingTransaction schema * #603 delete multisig tx util * #603 add sign command to support multisig development * #603 add --sender-address flag for generating unsigned bytes * #604 add sender address to unsigned tx * #603 return raw error from execute transaction to enable sqlite error casting * #603 complete multisig implementation * #603 fix tests * #603 add latest flag to pending transaction * #603 update pending_transaction status on executed * #603 add latest flag to pending transaction * #603 add filter to pending transaction query * #603 add latest flag to multisig-info and signatureInfo * #603 add multisig-child flag to tx table * #603 add multisig-child flag to tx table * #603 add multisig-child flag to tx model * #603 add latest field to all pending tables * #603 adjust query to handle latest flag * #603 insert multisig-child transaction to transaction table * #603 dfix test * #603 adjust cmd tools and crypto package to validate multisig signature (hierarchical multisig) * #603 fix example cmd * #603 update schema * #603 remove binary * #603 convert address-Signature to format stringTostring * #603 fix query inconsistency * #603 handle unsigned transaction execution status * #603 revert unnecessary change * #603 handle submit multisig info execute multiple tx * #603 fix query inconsistency * #603 add multisig signature type * #603 add constant for multisig signature validation type; * #603 remove debug log
* fixed invalid config * change image version to stretch * remove configuration for test purposes
* fixed typo
* add rate limiting feature for API * fix unclosed rows * change MaxAPIRequestPerSecond from int32 to uint32 * add download process metrics tracker * add go sum
* no need to validate current block height * fixing bug, respecting expiring escrow process on PushBlock * fixed escrow rollback query and respecting escrow stage before push block process * fixed escrow unit test result * has been fixed forgot to close rows * allow expiring escrow method to control db transaction
…g blocks from peers (#631) * Bug fixing * Add (dummy) ValidatePayloadHash to be used to validate payload data when downloading a block + attached data from other peers * Add GetPayloadHash and separated logic to compute block's payload hash from GenerateBlock to an individual method to make it reusable (eg. in GenerateBlock and ValidatePayloadHash) * Fix tests * Code Refactoring * Refactored method's name from GetPayloadHash to GetPayloadHashAndLength * Added logic to validate block's payload on block push * Add ValidatePayloadHash logic to blockMainService and blockSpineService * Fix tests * Add logic to validate spine block manifest against its relative spine block of reference * Add GetBlockFromTimestamp to blockQuery to be able to get the spine block of reference for a spine block manifest (which is the first block with timestamp >= spineBlockManifest.ExpirationTimestamp) * Add blockSpineService-specific method ValidateSpineBlockManifest to validate a spine block manifest after is downloaded from the p2p network * Add unit tests for new methods * Merge remote-tracking branch 'origin/develop' into 627-spine-block-manifest-publickeys-validation # Conflicts: # core/service/blockMainService.go * Code Refactoring * Update genesis block ID after changes to Generate block (added published receipts to payload length) * Add unit test for validateblockhash * Code Refactoring * Add 'dummy' interface BlockServiceSpineInterface to collect all signatures of spine block-specific methods * Code Refactoring * Removed unnecessary comment * Bug fixing * Remove fmt.println probably left from some internal test * Bug fixing * Update comment in query * Code Optimization * Optimize if statement by reversing conditions (courtesy of @ali) Co-authored-by: stefano galassi <[email protected]>
* Fixed bug in download snapshot process causing segfault Co-authored-by: stefano galassi <[email protected]>
* Fix bug in pendingSignatureQuery rollback function * Remove unused function from transactionQuery Co-authored-by: stefano galassi <[email protected]>
* no need to validate current block height * escrow query test fail fixed
* Fix bug in unchecked map key leading to panic in block smith process
* no need to validate current block height * escrow query test fail fixed * fixed escrow map params need to sorting before compare
add index on block table and update sql connnection number (#729)
scale down the common block jump (#731)
* accountDatasetsQuery should a plural name * update affected services and more about account_dataset stuff * fixing unit test of NewSnapshotFile integration * escrow validation timeout limit exceeded
* Code Refactoring * Add GetBlocksmithSafeQuery to query.go to retrieve a map with all queries that must save partial history to snapshots to not break blocksmith process * Refactor NewSnapshotFile (snapshot creation function) to use GetBlocksmithSafeQuery and get correct snapshot 'fromHeight' parameter * Refactor nodeRegistrationQuery 'SelectDataForSnapshot' to retrieve: * latest state from block 0 to 'fromHeight' * full history fromHeight to toHeight * merge and remove duplicate ids in first selection * Remove unused constant LinkedReceiptBlocksLimit Co-authored-by: stefano galassi <[email protected]>
…up with peers (#735) * making the download process less restrictive to failures in catching up with peers * fix the way of tracking errors * fix the way of keeping track the error
* TestBlockIncompleteQueueService_AddTransaction * add succes case
* fix return type to use blocker * add more blocker type
* update golangci to 1.24 * add reset db variable step * fixed invalid space * test circlleci building a branch * fixes golangci warn and error * disable ineffassign * adjust the deployment step
* fix error casting * revert p2p error type * fix blocker usage
* immediately release rows after converted to model * add unique index to published_receipt.datum_hash * fix: wrong column name * temporary fix, omit the unique
* NewBlockPoolService, GetBlocks, GetBlock, InsertBlock, ClearBlockPool * add assertion * gofmt -s -w
* enhance sign function & cmd signature * update readme * add option params on GenerateSignedTxBytes * update comment function SignEd25519 * add return err on parseBytesArgs func * fix type & unused arg * missing flag ed25519UseSlip10
handle error from block push and validation (#751)
andy-shi88
approved these changes
Apr 14, 2020
sukrawidhyawan
approved these changes
Apr 14, 2020
capt4ce
approved these changes
Apr 14, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enhancements:
Bug Fixes: