Skip to content

Commit cede23d

Browse files
astaphobiailtogaandy-shi88sg2bczsukrawidhyawan
authored
Bug fixes and tidying up (#799)
* Code Refactoring (#587) * Add spine blockquery to blocksmithstrategyspine to be able to get the previous spine block when selecting/sorting blocksmiths * Get escrow transactions with filter fields and pagination (#590) * get escrow transactions with filter fields and pagination * get escrow transactions functional test * #588 decouple published receipt and participation score from block main service (#591) * 592 fee model (#595) * #592 add fee model to action type * #592 add fee model to action type * #592 ignore package-interface naming lint rule * #592 update pb file of transaction : add model + service for getTxMinimumFee api * #592 add handler for calculate minimum fee api * #592 fix GetTransactionMinimumFee request * #592 fix wrong type * #592 add api client for testing get minimum fee * 578 snapshot file creation (#593) * Logic for snapshot file creation * Add new interface and service to abstract snapshot file creation for different block types (for now only mainchain) * Add ugorji/go/codec package to add CBOR serialization support * Added Logic for snapshot file processing (create/download) * Add new interface and service to abstract snapshot file download * Implemented some of the logic in snapshotMainBlockService for downloading and creating new snapshot files/chunks * Added business logic for snapshots * Add new service FileService to abstract (and make testable) methods relative to file manipulation * Better separation of logic between shapshotservice (generic methods) and snapshotBlockService (blocktype-specific methods) * Move GetFileNameFromHash and GetHashFromFileName into fileservice * Add query and service handler to retrieve account balances for snapshots * Add minimal process management for snapshot files: if a snapshot of the same block type is still running and a new one is started, the new one will stop the old one (this logic must be discussed and is a WIP) * Add initial logic to create a snapshot file containing all (only last state) account balances * TODO: increase code coverage (few methods are tested so far) * Bug fix * Add EscrowTransactionQuery to GetDerivedQueries * Added logic for generating snapshot files * Add All relevant queries and wrappers (to ease testing) to snapshotMainBlockService * Add test case for NewSnapshotFile and new queries * Finalised logic for generating snapshot files (first iteration only has single file per snapshot) * Add all relevant queries to get snapshot's payload * Add method to parse and import snapshot into db * Add config parameter for snapshots folder * Add unit and integration tests * Added logic for download snapshots * Add method to parse the spineBlockManifest and download file chunk hashes * TODO: download file is still mocked * Uncomment logic previously commented for testing purposes * Increase MainchainSnapshotInterval value to 1440 to avoid having too many snapshots in dev and production environments * Add query interface for snapshot-specific queries (SnapshotQuery) * Add SelectDataForSnapshot interface signature to new interface * Add GetSnapshotQuery method to get all snapshot queries implementations (in a map) * Refactor relative query methods in every relevant repository * Refactor some constant to use time.Duration instead of int type * Fix unit tests * Add DESC to order by clauses in snapshot queries * Refactor GetChainTypeCount to GetChainTypes to make chain types countable * Remove temporary test function * Use SnapshotQuery interface to loop through all snapshot query implementations instead of adding them one by one (less code duplication) * Use SnapshotQuery interface to loop through all snapshot query implementations instead of adding them one by one (less code duplication) * 596 multisig skeleton (#597) * #596 add multisignature transaction body * #596 add constants for MultiSignature body fields * #596 add multisignature tx type skeleton * #596 add multisig tx type enum * #596 add command tools for generate multisignature tx and parsing any tx * #596 basic multisignature parsing functionality * #596 update schema commit * #596 add signature info * #596 update signature structure to use map * #596 adjust cmd tools to new multisig byte structure * #596 update multisig transaction bytes * #596 update schema * 599 snapshot chunk strategy (#605) * Added proto message for snapshots * Add SnapshotPayload message representing internal snapshot data structure * Added logic for snapshot chunk 'basic' strategy * Add Strategy (interface and implementation) for 'basic' snapshot chunks: chunks are generated by splitting a full encoded snapshot * Add SnapshotPayload model to schema, since we are using it in two different packages * Refactor snapshotmainblockservice to use new snapshot chunk strategy * Refactor unit and integration tests * Completing logic for snapshot basic chunk strategy * Finalize BuildSnapshotFromChunks and its test * Add ReadFileByHash to FileService to be able to directly read a file from its hash-name and mock ioutil.ReadFile in services that depend on this service * Remove unused parameter 'chunkSizeBytes' from SnapshotBlockServiceInterface.NewSnapshotFile * 600 multisig queries (#607) * #600 add new managed table for multisig * #600 update multisig managed table model * #600 add managed table query for multisig Co-authored-by: Ariasa Ketut <[email protected]> * 602 generate multisig address (#609) * method multisig account address assembler * command line tool multisig generator command * set 1.20.0 of golangci-lint version to fixed circleci fail * no need to validate current block height (#610) * inhibit spine blocks smithing process during main blocks first download (#608) * Refactor spineBlockDownloadService to blockStatusService, to be able to set/get all chaintype blockchain statuses (for now only if a node has finished downloading the block first time after has started) * Refactor downloadBlockchain to use blockStatusService as dependency and set blockchain download status referenced by blockStatusService * Simplify main.go by moving logic to set first blockchain download into downloadBlockchain service * Add new blocker type P2PNetworkConnectionErr * Increase timers for checking block download process * Refactor blockStatusService and implementations to blockTypeStatusService * Refactor relative code in other services * Remove cast to int snapshotPayloadHeight in snapshotMainBlockService * Fix bug in snapshotPayloadHeight computation * Refactor constructor method for BlockchainSyncService * Fixed transaction generator affected by fee (#611) * no need to validate current block height * fixed transcation generator afftected by fee * Code Refactoring and Bug Fixing (#613) * Fix logic for unlocking smithing process based on network status and blockchain download status (all main blocks downloaded) * Add new service variable and relative getter/setter to blockTypeStatusService 'isSmithingLocked' to control whether smithing is locked or not, regardless of the blockchain download status * Remove unused getter from chainType * Add new constants for better code readability * Fix tests * 601 multisig validation (#615) * #601 add --post and --post-host to enable post transaction bytes on generated * #601 add multisig validation * #601 fix lint * #601 wrong description in cmd * Optimize GRPC duration (#616) * no need to validate current block height * set index into some tables for speed up latency get a lot of records * remove unused a line of codes * GRPC endpoint get single escrow transaction by ID (#620) * no need to validate current block height * functionality test get escrow by id * Download snapshot chunks (#614) * Change fatalf to errorf/return in goroutine in snapshotService.go * Add missing return statement for error in snapshotService.go * Added logic for downloading files from a random peer and Code Refactoring * Refactor filedownloader: moved to p2p package because being in service generates a cycle dependecy * Add rpc call (client request and server response) to download a file from a random peer * Add logic to download/parse/verify a file and a snapshot * Added logic and file download metrics * Add logic to download a file, server side * Add metrics to count succeeded/failed snapshot downloads * Update schema: add Failed files for file download * Add metric IncrementSnapshotDownloadCounter to count number of downloaded files from every node * Refactor InsertSpineBlockManifest to not fail if a new record with same id is inserted, to allow downloaded spine block manifests to override locally (probably wrong) generated ones * Rename megablockQuery to spineBlockManifestQuery * Refactor blockchainStatusService to use safer (for r/w race conditions) and faster sync.Map instead of regular map + sync.mutex * Fix bug in spineBlockService.push: insert spineBlockManifest records if they are present as part of block data * Fix smith process lock when downloading spine and main blocks * Fix download blockchain process: spine blocks, snapshot, rest of main blocks. then start smithing * Add retry download failed file chunks from different peers * WIP: commented code to make it easier for others to run test nodes locally and double check the download process works as expected. All commented code is prepend with "// STEF" string * Simplified fileService GetFileNameFromHash and GetFileNameFromBytes so that they don't have to return error and refactored relative code that uses them * Add basic unit test for p2p.DownloadFilesFromPeer (still needs more test cases) * Optimize p2p.DownloadFilesFromPeer while writing the test * Fix code after code review * fixed bug in caclulating snapshot's height * Add two struct and methods to mimic sync.Map behaviour, but with regular map + sync.RWMutex (more efficient on machines with low cpu core count) * Refactor code that used sync.Map in favor of new map struct * Add util function to calculate a quick (non secure) random number * Fix uncaught nil in p2pServerHandler * Add NewSpineBlockManifestQuery to derivedqueries to make it rollback safe (it must be rolledback together with mainchain blocks) * Add a couple of TODOs for spine block manifest validation * Fix bug in unitialized model.MapIntBool struct * Add missing test for GenerateSnapshot method * Uncommented code for testing * Uncomment code * Fix one unit test after fixing the snapshot height (previous commit) Co-authored-by: stefano galassi <[email protected]> * 594 add bitcoin signature (#618) * add bitcoin sig, btcd & btcutil package * restructure Ed25519 signature type function * add cmd to generate bitcoin account * add signature-type flag in cmd tx generator * cmd enhace switch case generate account * return error when sign and verify payload * cmd generate tx with sender signature type * adjust general parse transaction bytes * adjust unit test * udate cmd readme * cmd, panic when invalid signature type * remove unused function * add generate account in signature * add signature unit test * add misisng import package * update schema * change parameter type of GenerateAccountFromSeed Co-authored-by: Ariasa Ketut <[email protected]> * 629 - Use gomod consistently (#630) * no need to validate current block height * update circleci config use go modules instead of dep and update go version to 1.14 * go mod and more cleaning shadow declaration variable * Revert "GRPC endpoint get single escrow transaction by ID (#620)" This reverts commit 7824924. * Revert "Revert "GRPC endpoint get single escrow transaction by ID (#620)"" This reverts commit bedbf87. * revert unused commit * remove Gopkg.* files * remove no needed comment * make sure go modules cache correctly * change circleci working directory, update readme * update path of cache * fixed go modules caches * 603 multisig apply (#623) * #603 initialize multisig service layer * #603 implement apply confirmed * #603 add sender address in pending transaction * #603 update multisignature.PendingTransaction schema * #603 delete multisig tx util * #603 add sign command to support multisig development * #603 add --sender-address flag for generating unsigned bytes * #604 add sender address to unsigned tx * #603 return raw error from execute transaction to enable sqlite error casting * #603 complete multisig implementation * #603 fix tests * #603 add latest flag to pending transaction * #603 update pending_transaction status on executed * #603 add latest flag to pending transaction * #603 add filter to pending transaction query * #603 add latest flag to multisig-info and signatureInfo * #603 add multisig-child flag to tx table * #603 add multisig-child flag to tx table * #603 add multisig-child flag to tx model * #603 add latest field to all pending tables * #603 adjust query to handle latest flag * #603 insert multisig-child transaction to transaction table * #603 dfix test * #603 adjust cmd tools and crypto package to validate multisig signature (hierarchical multisig) * #603 fix example cmd * #603 update schema * #603 remove binary * #603 convert address-Signature to format stringTostring * #603 fix query inconsistency * #603 handle unsigned transaction execution status * #603 revert unnecessary change * #603 handle submit multisig info execute multiple tx * #603 fix query inconsistency * #603 add multisig signature type * #603 add constant for multisig signature validation type; * #603 remove debug log * CI delivery image version (#632) * fixed invalid config * change image version to stretch * remove configuration for test purposes * Ci delivery image version (#633) * fixed typo * 604 api rate limit (#625) * add rate limiting feature for API * fix unclosed rows * change MaxAPIRequestPerSecond from int32 to uint32 * add download process metrics tracker * add go sum * 621 - Respecting escrow expiration on Push Block (#626) * no need to validate current block height * fixing bug, respecting expiring escrow process on PushBlock * fixed escrow rollback query and respecting escrow stage before push block process * fixed escrow unit test result * has been fixed forgot to close rows * allow expiring escrow method to control db transaction * Spine block manifest and spine public keys validation when downloading blocks from peers (#631) * Bug fixing * Add (dummy) ValidatePayloadHash to be used to validate payload data when downloading a block + attached data from other peers * Add GetPayloadHash and separated logic to compute block's payload hash from GenerateBlock to an individual method to make it reusable (eg. in GenerateBlock and ValidatePayloadHash) * Fix tests * Code Refactoring * Refactored method's name from GetPayloadHash to GetPayloadHashAndLength * Added logic to validate block's payload on block push * Add ValidatePayloadHash logic to blockMainService and blockSpineService * Fix tests * Add logic to validate spine block manifest against its relative spine block of reference * Add GetBlockFromTimestamp to blockQuery to be able to get the spine block of reference for a spine block manifest (which is the first block with timestamp >= spineBlockManifest.ExpirationTimestamp) * Add blockSpineService-specific method ValidateSpineBlockManifest to validate a spine block manifest after is downloaded from the p2p network * Add unit tests for new methods * Merge remote-tracking branch 'origin/develop' into 627-spine-block-manifest-publickeys-validation # Conflicts: # core/service/blockMainService.go * Code Refactoring * Update genesis block ID after changes to Generate block (added published receipts to payload length) * Add unit test for validateblockhash * Code Refactoring * Add 'dummy' interface BlockServiceSpineInterface to collect all signatures of spine block-specific methods * Code Refactoring * Removed unnecessary comment * Bug fixing * Remove fmt.println probably left from some internal test * Bug fixing * Update comment in query * Code Optimization * Optimize if statement by reversing conditions (courtesy of @ali) Co-authored-by: stefano galassi <[email protected]> * Bug Fixing (#636) * Fixed bug in download snapshot process causing segfault Co-authored-by: stefano galassi <[email protected]> * Bug Fixing (#639) * Fix bug in pendingSignatureQuery rollback function * Remove unused function from transactionQuery Co-authored-by: stefano galassi <[email protected]> * Escrow query test fail fixed (#640) * no need to validate current block height * escrow query test fail fixed * Bug Fixing (#642) * Fix bug in unchecked map key leading to panic in block smith process * Escrow query test fail fixed (#647) * no need to validate current block height * escrow query test fail fixed * fixed escrow map params need to sorting before compare * 634 multisig api (#645) * #634 init multisig api * #634 update multisig request model * #634 update get pending tx detail response * #634 update get pending tx detail response * #634 initiate multisig client API * #634 fix bug: wrong empty signature checks * #634 remove built binary from repository * #634 add multisig api tests * #634 revert unnecessary changes * #634 update go mod * #634 initiate health check service and model * #646 add health check endpoint * #634 update schema * #634 use getter instead of accessing field directly * #634 add stricter param check * #634 fix pagination * #634 remove duplicate code Co-authored-by: Ariasa Ketut <[email protected]> * 634 multisig api (#653) * #634 add default field to order field param Co-authored-by: Ariasa Ketut <[email protected]> * #651 fix multisig_address signature validation (#652) * #651 fix multisig_address signature validation * #651 fix test Co-authored-by: Ariasa Ketut <[email protected]> * 656 incorrect account ledger balance changes (#657) * no need to validate current block height * account ledger incorrect balance change information fixed * 654 account ledger multisig (#658) * #654 add multisig transaction event to account ledger * #654 add ledger record in multisig transaction * #654 update schema commit Co-authored-by: Gede Sukra Widhyawan <[email protected]> * 660 api get multisig info (#661) * #660 add new api endpoint proto for getting multisignature info * #660 implement get multisig info api * #660 rename getPendingTransactions request * #660 rename getPendingTransactions request * #660 update schema commit * 650 - CMD HD wallet generate account (#668) * no need to validate current block height * generate account slip10 * allow fetch from private repo * fixed invalid format of ci * fixed set env of machine * add additional ssh key * fixed bash condition when check gomod exists * allowed github connection via token instead of ssh * remove unused code * more describe generate account command * fix uninitialized metrics (#662) fix uninitialized metrics (#662) * bitcoin sig: add public key into signature (#670) * bitcoin sig: add public key into signature * handle multiple rows in a function (#675) handle multiple rows in a function (#675) * SelectReceipt Limit (#676) SelectReceipt Limit (#676) * Circleci GitHub token (#677) * token as env variable * 628 apply snapshot (#667) * added alpha nodes genesis.go * Rollback mempool - Restoring mempool transactions backup (#419) * initiate restoreMempoolsBackup func * finished mempool backup process and add loggger into processfork struct * add logger into blockPopper stuff * 420 get scramble nodes enhance (#423) * #420 rename get scrambled nodes function to get last scrambled node, since we will have multiple version now * #420 move some scrambling function to util to reduce fetching function call frequency * #420 add block_height to merkle tree table * #420 fix query to fetch receipt, it had wrong block height scope on the query * #420 handle error * #420 add receipt validation * #420 build scrambled node at height x * #420 adjust constructor update * #420 move receipt service instantiation to after node registration service * #420 move scramble node model outside, it is used by other service as well : receipt service * #420 validate receipt before storing to database * #420 use bytes.equal for better comparison performance * #420 rename memoized scramble nodes * #420 simplify loop * 424 fix add participation score query (#425) * fixed addParticipationScore query refactored unit test refactored nodeRegistration applyConfirmed to use addParticipationScore instead of insertParticipationScore for gnenesis block (so we can remove that unsafe query :) ) * removed unused InsertParticipationScore query * refactored addparticipationscore query to use a single upsert statement * Remove some process (#429) * disable some process to adjust with alpha * update unit test * reverted broken query and kept changes from newer pr * 432 revert participation score query (#437) * 424 fix add participation score query (#425) * fixed addParticipationScore query refactored unit test refactored nodeRegistration applyConfirmed to use addParticipationScore instead of insertParticipationScore for gnenesis block (so we can remove that unsafe query :) ) * removed unused InsertParticipationScore query * refactored addparticipationscore query to use a single upsert statement * Remove some process (#429) * disable some process to adjust with alpha * update unit test * reverted query * reverted broken participation score query * change a few configurations special for alpha (#434) * change a few configurations special for alpha * updated config.toml * added (peer) port to genesis nodes addresses * fixed test * updated production values for some constants * updated log levels for testing alpha nodes * reset default log levels * increased PriorityStrategyBuildScrambleNodesGap to 40 * fixed unwanted diffrences between develop and alpha branches * fixed unit tests for alpha * fixed unit test for mempoolCoreService * Fixed unit test * Commented out startSpinechain, to inhibit spine blocks generation: at this stage it won't make any difference, since we need snapshots to make a use of spine blocks * switch log level for debugging purpose * debug: process fork warn to errorf log * revert: revert log levels * keep track of p2p go routine metrics (#569) * fix shadowed error message * Resolved merge conflicts * Code refactoring * Updated mainchain genesis id * Code Refactoring * Update main block genesis ID * Updated constant * Update constant DownloadSnapshotNumberOfRetries to be = MaxResolvedPeers so that if a file fails to download the node will cycle trough all resolved peers until it is downloaded or it fails * Update golang.org/x/crypto/ed25519 to crypto/ed25519 * Reverted crypto/ed25519 to golang.org/x/crypto/ed25519 * Fixed bug in download snapshot process causing segfault * Adding business logic * Add SnapshotQueries for pendingsignature, pendingtransaction, multisiginfo * Add logic to snapshot service to include multisig data in snapshots * Added business logic * Add TrimDataBeforeSnapshot to all SnapshotQueries * Added business logic * Add TrimDataBeforeSnapshot to all SnapshotQueries * Refactor snapshot file downloader to return snapshotFileInfo (ready to be read/imported) * Add logic to main.go to import snapshot after downloading it * Refactor IsSnapshotHeight with simplified logic * Refactor code to be aligned with develop branch instead of alpha * Refactor SelectDataForSnapshot queries to use rollback-like logic * Refactor SelectDataForSnapshot queries to use rollback-like logic * Write unit tests for new queries * Fix wrong table nale in new block query * Fix wrong table nale in new block query * Fix do not allow to delete genesis block when trimming block table * Revert configX.toml log level to "fatal", "error", "panic" * #628 add checks if all blocksmith time has expired * #628 move push block to anonymous function * Add rollback to snapshot height to set latest = 1 to all latest entitites * Fixed SelectDataForSnapshot query in accountDataset repo * Fix missing for loop break conditions in ScanBlockPool * Add function to log running binary file checksum at node startup Co-authored-by: stefano galassi <[email protected]> Co-authored-by: Andy Shi <[email protected]> Co-authored-by: Ahmad Ali Abdilah <[email protected]> Co-authored-by: stefano galassi <[email protected]> * 672 makefile binary built (#678) * #672 ignore release folder * #672 initialize makefile * #672 add github token example file * #672 move xgo to zoobc repository * #672 add makefile usage * #672 add cc for cmd * #672 add go-fmt and separate golangcilint version * #672 move test to makefile command * #672 fix gopath * #672 update readme for github.token usage * #672 print information for step ran Co-authored-by: Ahmad Ali Abdilah <[email protected]> * 665-Enhance CMD Database Init (#680) * cmd: put init db only where is needed * cmd: add commad to generate proof of ownership * cmd: add readme rollback * cmd: fixing remove node reference func * handle nil value (#682) * throw error on over limit get blocks (#684) * throw error on over limit get blocks * move limit validation to handler layer * remove pending tx status change, it has been updated in apply confirmed (#685) * 686: Cmd, enhance generate account (#687) * cmd: enhance generate account * add get public key using slip10 * 655 escrow receipt approval (#688) * no need to validate current block height * account dataset escrow approval and integration * fullfill send money validation test cases * to be gentle no need to check the account dataset property * create specific method of query for account dataset * getHostInfo unit testing (#689) * getHostInfo unit testing Co-authored-by: Nawi Kartini <[email protected]> Co-authored-by: Ariasa Ketut <[email protected]> Co-authored-by: Ahmad Ali Abdilah <[email protected]> * adding database status metrics (#693) adding database status metrics (#693) * Bug Fixing snapshot selection queries and enhancing rollback queries (#696) * Bug Fixing * Fix snapshot selection queries * Enhance rollback queries * Enhance db tables by adding composite primary keys where possible * Fix bug in rollback and snapshot selection queries Co-authored-by: stefano galassi <[email protected]> Co-authored-by: Andy Shi <[email protected]> * 681 disable smithing non registered node (#694) * #681 add zero participation score blocker * #681 inject blockchainStatusService to noderegistration service * #681 add flag to say if a node is blocksmith or not * #681 update main.go with new struct constructor * #681 only update the blocksmith status when detect zero participation score * #681 compare with bytes.equal() * unit testing for GetBlockQueue (#698) unit testing for GetBlockQueue (#698) * Bug Fixing (#706) * Fix breaking bug in rollback query * Refactor spinePublicKey rollback query to comply to new code logic * 691 grpc account dataset (#699) * no need to validate current block height * rename accountDataset to plural name * account dataset grpc service list and single record * grpc client example * move params checkup to handler package * remove unnecessary test cases * technical error message passed * 690 empty block smith (#703) * #690 add empty flag to enable empty block generate * #690 add limit for skipped block smith * #690 set the skipped limit back to 15 * #690 remove unimplemented test * #690 fix return arguments * #690 remove unnecessary variable initialization * #690 make use of default int64 value * #690 smaller skipped blocksmith constant so we can see the effect on dev net * adding function for HD wallet testing (#683) * adding function for HD wallet testing * P2P: Add unit test on server service (#707) * p2p: add unit test p2p server service * enhance Err format * change GetPeerInfoResponse field name * p2p: update unit test * update schema Co-authored-by: Andy Shi <[email protected]> * added skippedblocksmith to snapshot (generate/apply) (#710) * Add SkippedBlocksmith to snapshot proto model * Added Logic for snapshots * Add selectDataForSnapshot and TrimDataBeforeSnapshot queries to skippedBlockSmithQuery repository * Added Logic for snapshots * Add logic to include skippedblocksmith records to snapshots * Add logic to import skippedblocksmith records from snapshots into db (apply snapshot) * Fix unit tests * Use getter method instead of direct variable in skippedBlocksmithQuery TrimDataBeforeSnapshot Co-authored-by: stefano galassi <[email protected]> * Adding runtime cpu profiling for core node (#712) * Adding Application Logic * Add configuration parameter for profiling * Add runtime flag to enable profiling * Add logic to run cpu profiling * Update configuration parameter for profiling * Add logic to enable memory profiling * Update documentation (readme.md): see 'run' paragraph * add more info in log (#713) * fixed inconsistance block height on escrowed transaction (#717) * Code Refactoring (#716) * Use net/http/pprof package that serves the full profiling api via http * Enable profiling api only when running app with -cpu-profile flag (see updated readme) * Refactor the way monitoring server is instantiated: not using default http hanlder anymore * Update readme.md Co-authored-by: stefano galassi <[email protected]> * Code Refactoring (#719) * Refactor block query to select a range of blocsk instad of a single one for snapshots * Refactor service logic to add more blocks (720) to snapshots * Cleaning redundant function (#714) * adding function for HD wallet testing * cleaning some unused variables * remove unused const and redundant function on generator * Bug Fixing (#724) * Fix pprof http server listening only on localhost Co-authored-by: stefano galassi <[email protected]> * Code Refactoring (#728) * Optimize function GenerateTxRemoveNode Co-authored-by: stefano galassi <[email protected]> * 722 blockchain orchestrator (#726) * add BlockchainOrchestrator to control the chains * 709 Smithing All Blocksmith Skipped (#725) * #709 add smithing pending blocker * #709 remove smith time from blcoksmith model * #709 update smithing process: recycle blocksmith when all blocksmith time are passed * #709 separate smith time constants between spinechain and mainchain * #709 inject spinechain blocksmith service * #709 update metric smithtime to smith index * #709 cover new smithing process unit tests * #709 fix comment on blocksmith strategy & update fake smith function signature * add index on block table and update sql connnection number (#729) add index on block table and update sql connnection number (#729) * scale down the common block jump (#731) scale down the common block jump (#731) * 721 - Remove Account Dataset Expiration (#732) * accountDatasetsQuery should a plural name * update affected services and more about account_dataset stuff * fixing unit test of NewSnapshotFile integration * escrow validation timeout limit exceeded * Add node registry partial history to snapshots (#734) * Code Refactoring * Add GetBlocksmithSafeQuery to query.go to retrieve a map with all queries that must save partial history to snapshots to not break blocksmith process * Refactor NewSnapshotFile (snapshot creation function) to use GetBlocksmithSafeQuery and get correct snapshot 'fromHeight' parameter * Refactor nodeRegistrationQuery 'SelectDataForSnapshot' to retrieve: * latest state from block 0 to 'fromHeight' * full history fromHeight to toHeight * merge and remove duplicate ids in first selection * Remove unused constant LinkedReceiptBlocksLimit Co-authored-by: stefano galassi <[email protected]> * making the download process less restrictive to failures in catching up with peers (#735) * making the download process less restrictive to failures in catching up with peers * fix the way of tracking errors * fix the way of keeping track the error * TestBlockIncompleteQueueService_AddTransaction (#720) * TestBlockIncompleteQueueService_AddTransaction * add succes case * fix return type to use blocker (#742) * fix return type to use blocker * add more blocker type * Update golangci to 1.24.0 (#743) * update golangci to 1.24 * add reset db variable step * fixed invalid space * test circlleci building a branch * fixes golangci warn and error * disable ineffassign * adjust the deployment step * #740 index receipt fields for better query time (#744) * #740 index receipt fields for faster query * #740 verbose message on golangcilint run * fix error casting (#747) * fix error casting * revert p2p error type * fix blocker usage * downgrade golangci-lint version (#748) * Hotfix rows immediate release (#749) * immediately release rows after converted to model * add unique index to published_receipt.datum_hash * fix: wrong column name * temporary fix, omit the unique * Unit Testing core service BlockPoolService (#739) * NewBlockPoolService, GetBlocks, GetBlock, InsertBlock, ClearBlockPool * add assertion * gofmt -s -w * Add optional Params on Sign function (#746) * enhance sign function & cmd signature * update readme * add option params on GenerateSignedTxBytes * update comment function SignEd25519 * add return err on parseBytesArgs func * fix type & unused arg * missing flag ed25519UseSlip10 * handle error from block push and validation (#751) handle error from block push and validation (#751) * fix block validation and smith time calculation (#752) * setter and recipient must be the same person (#755) * Fix: P2P - Can't Add UnresolvedPeers (#757) * add condition when there are still space in unresolved peers * remove duplicate removal query (#758) * Optimize database transaction (#756) * use ExecuteSelectRow instead * use QueryRow instead of Query, get single account balance * use QueryRow instead if only want a single record * nodeReg.ApplyConfirmed better to use QueryRow * nodeRegUpdate.ApplyConfirmed better to use QueryRow * removeNodeReg.ApplyConfirmed better to use QueryRow * CMD fix and addition (#762) * add blockheight logging at the end of applying a snapshot * fix rollback cmd method to use the parameter passed * add cmd methods for getting scrambledNodes and priorityPeers * rename folder of scrambled nodes * Use QueryRow instead of Query when getting single record (#764) * removeNodeReg.Validate better to use QueryRow * nodeRegistrationUpdate.ApplyUnconfirmed better to use QueryRow Co-authored-by: Andy Shi <[email protected]> * fix query and the unit tests (#765) fix query and the unit tests (#765) * Confirm the state of download finish if the whole network stuck (#770) * confirm the state of download finish if the whole network stuck * reset the firstDownloadCounter when finding out they are not stuck * reduce the validation limit constant * Unit Test Transaction Core Service (#767) * UndoApplyUnconfirmedTransaction * ApplyConfirmedTransaction * ApplyUnconfirmedTransaction * ValidateTransaction * remove underscores in Go names Co-authored-by: Nawi Kartini <[email protected]> Co-authored-by: Ahmad Ali Abdilah <[email protected]> * GetBlockByHeight: Query row instead of Query (#768) * removeNodeReg.Validate better to use QueryRow * nodeRegistrationUpdate.ApplyUnconfirmed better to use QueryRow * GetBlockByHeight better to use QueryRow Co-authored-by: Andy Shi <[email protected]> * fix finish download (#771) fix finish download (#771) * reset db automatically and also snapshots stuff (#772) * reset db automatically and also snapshots stuff * *_kv would remove also * snapshots suffix * Fix manifest rollback (#773) * add spineblock height field * rename spine block manifest column and add indesx on block height column * rename spine_block_manifest column field * move spine_block_manifest to spinechain derived query category * clean snapshot file on rollbacked * add delete snapshot file function * update tests * remove ErrNoRows check as that error only occur on sql.Row not sql.Rows * inject snapshot block service to spine block service * snapshot manifest field rename * fix new rollback query for manifest * use parse instead of splitting chunks manually * log the deletion error * handle error * update schema * add extra comment for testing * move file deletion to after db commit and as goroutine Co-authored-by: Ahmad Ali Abdilah <[email protected]> * #777 update validate block function signature : remove timestamp (#779) * #777 update validate block function signature : remove timestamp * move the validation of download finish if the block height is 0 (#781) move the validation of download finish if the block height is 0 (#781) * Bug limit receipt selection when select by root 774 (#776) * versi 1 * small edit * update mock ExecuteSelect * make sure the query is similar as mock * make it simpler * merge schema Co-authored-by: Nawi Kartini <[email protected]> * 782 missing blocks fix (#783) * #782 validate block in scanBlockPool thread * #782 fix wrong function signature causing interface not implemented * #782 move chain lock to beginning of ScanBlockPool function Co-authored-by: Ahmad Ali Abdilah <[email protected]> * Getting own public ip (#784) * get own public ip * allowing to get public ip via checkip.dyndns.org * make sure the parsed ip is valid ip address * IPUtil methods * Fixing error nil pointer when accessing sql.Tx Executor (#786) * add missing err checker on validate mempool * fixing racing unlocking mutex & making sql.Tx nil * Sort poppedBlocks by height ascending (#787) * sort poppedBlocks by height desc * fulfill test cases of PoppedoffBlocks * sort ascending popped off blocks * fulfill spine PopOffToBlock test cases * #792 replace spendable with balance (#793) * Genesis generate command: Code Refactoring and adding new feature to facilitate deployments with nomad/consul (#797) * Code Refactoring and adding new feature * Refactor command code to comply to new standards (see other commands) * Add feature to generate consulKvInit.sh script from a template and a the newly generated genesis * Bug fixing * Limiter must be 2*MinRollbackHeight (#795) * limiter must be 2*MinRollbackHeight * 789 multisig pending transaction Apply-Flow (#798) * #789 update pending_transaction query by hash to have multiple status filter * #789 update flow of pending_transaction execution * Tech: GetAccountBalances #785 (#788) * first commit * check num of addresses * fix * fix * fix * fix the unit testing * use caseQuery instead of looping * fix caseQuery.In * fix unit testing * move validation to handler * fix unit test part 1 * auto update from proto * accountBalance * change to status.Error * change to plural * refine according PR comment * fix * fix unit test part 1 * remove commented line Co-authored-by: Nawi Kartini <[email protected]> Co-authored-by: Ariasa Ketut <[email protected]> * fix the status for easier debug (#803) fix the status for easier debug (#803) * Bug: Limit Receipt Selection When Select By Root (#808) * change 'where limit' on sql * UnitTest-ApiService-GetBalances (#804) * first commit (#807) unit test on api handler `GetAccountBalance` * #800 do mempool backup before executing derived query Rollback() method (#810) * remove key from badger db after mempools backup restored (#811) remove key from badger db after mempools backup restored (#811) * add badger metrics to prometheus (#815) * add badger metrics to prometheus * explicitly use badger v2 * remove badgerdb v1 * revert MaxAPIRequestPerSecond to avoid nodes being overwhelmed by api calls * enhance error logging for pushBlock and validateBlock process (#818) * enhance error logging for pushBlock and validateBlock process * exclude unimportant push and validate block error * change block_height to reference_block_height (#820) change block_height to reference_block_height (#820) * Bug: Missing Rollback When BackupMempool Fail in PoppedOffBlock #802 (#816) * first commit * fix: Missing Rollback When BackupMempool Fail in PoppedOffBlock * change BackupMempols --> BackupMempools * Unit test api handler account dataset (#821) * GetAccountDatasets * GetAccountDataset * Expire musig pending transaction (#828) * ignore .bak like genesis.go.bak :) * expiring pending transactions by height and undo apply unconfirmed * update readme about multis sig generator * not allow filter by pending and expired * finish query logic to, so now (block_height+min_rollback) = current_block_height * defer func for rows close and matches the others about query logic * add log instance into transaction core service, ignore log level Co-authored-by: Gede Sukra Widhyawan <[email protected]> Co-authored-by: Andy Shi <[email protected]> * Unit test api block handler (#830) * TestBlockHandler_GetBlock * TestBlockHandler_GetBlocks * fix captLocal * fix gocritic * Unit test api mempool handler (#831) * TestMempoolTransactionHandler_GetMempoolTransaction * TestMempoolTransactionHandler_GetMempoolTransactions * 829 download escrow transaction (#834) * #832 add one more persistance time cycle to check to avoid node stuck in small network * #829 include escrow detail of transaction in GetTransactionByBlockID * #829 include escrow detail of transaction in GetTransactionByBlockID * #833 add copy of block to broadcast before deleting the transactions * #829 fix test * #829 add test for escrow transaction query * #833 broadcasted wrong variable * #829 refactor escrow transaction query to be more generic * Hotfix deepcopy broadcast block (#837) * add mohae/deepcopy dependency * enforce deep copy for broadcast block * use map to reduce loop complexity (#838) * get the latest account balances (#839) Co-authored-by: iltoga <[email protected]> Co-authored-by: Andy Shi <[email protected]> Co-authored-by: stefano galassi <[email protected]> Co-authored-by: Gede Sukra Widhyawan <[email protected]> Co-authored-by: Ahmad Ali Abdilah <[email protected]> Co-authored-by: Nawi Kartini <[email protected]> Co-authored-by: stefano galassi <[email protected]> Co-authored-by: Kevin Hadi <[email protected]> Co-authored-by: Nawi Kartini <[email protected]>
1 parent 8d632d7 commit cede23d

File tree

106 files changed

+4385
-1741
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

106 files changed

+4385
-1741
lines changed

.circleci/config.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,22 @@ reset_dbs: &reset_dbs
77
name: RESET DBs
88
command: |
99
if [[ ${CIRCLE_BRANCH} == *"reset-db"* ]]; then
10-
for host in $ln1 $ln2 $ln3; do ssh root@$host 'for f in *.db; do mv -- "$f" "_$f"; done'; done
10+
for host in $ln1 $ln2 $ln3; do
11+
echo "Trying to reset db on $host"
12+
ssh root@$host 'cd /root/zoobc/resource && for f in *.db; do mv -- "$f" "_$f"; done && rm -rf snapshots* *_kv';
13+
done;
14+
else
15+
echo "All good without reset db";
1116
fi
1217
reset_db: &reset_db
1318
name: RESET DB
1419
command: |
1520
if [[ ${CIRCLE_BRANCH} == *"reset-db"* ]]; then
16-
ssh root@$ln4 'for f in *.db; do mv -- "$f" "_$f"; done';
21+
echo "Trying to reset db on ${ln4}"
22+
ssh root@$ln4 'cd /root/zoobc/resource && for f in *.db; do mv -- "$f" "_$f"; done && rm -rf snapsnots* *_kv';
23+
else
24+
echo "All good without reset db"
1725
fi
18-
1926
jobs:
2027
prepare:
2128
<<: *defaults

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,12 @@ resource/zoobc_*
1919
resource/snapshots*
2020
resource_cluster/zoobc_*
2121
resource/*.prof
22+
resource/generated
2223
cmd/*.new
2324
accounts.txt
2425
.editorconfig
2526
.manual
2627
release/
2728
github.token
29+
*.back
30+
*.bak
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"fmt"
6+
7+
log "github.com/sirupsen/logrus"
8+
"github.com/spf13/viper"
9+
rpc_model "github.com/zoobc/zoobc-core/common/model"
10+
rpc_service "github.com/zoobc/zoobc-core/common/service"
11+
"github.com/zoobc/zoobc-core/common/util"
12+
"google.golang.org/grpc"
13+
)
14+
15+
func main() {
16+
var apiRPCPort int
17+
if err := util.LoadConfig("../../../resource", "config", "toml"); err != nil {
18+
log.Fatal(err)
19+
} else {
20+
apiRPCPort = viper.GetInt("apiRPCPort")
21+
if apiRPCPort == 0 {
22+
apiRPCPort = 8080
23+
}
24+
}
25+
26+
conn, err := grpc.Dial(fmt.Sprintf(":%d", apiRPCPort), grpc.WithInsecure())
27+
if err != nil {
28+
log.Fatalf("did not connect: %s", err)
29+
}
30+
defer conn.Close()
31+
32+
c := rpc_service.NewAccountBalanceServiceClient(conn)
33+
34+
response, err := c.GetAccountBalances(context.Background(), &rpc_model.GetAccountBalancesRequest{
35+
AccountAddresses: []string{
36+
"OnEYzI-EMV6UTfoUEzpQUjkSlnqB82-SyRN7469lJTWH",
37+
"BCZEGOb3WNx3fDOVf9ZS4EjvOIv_UeW4TVBQJ_6tHKlE",
38+
"iSJt3H8wFOzlWKsy_UoEWF_OjF6oymHMqthyUMDKSyxb",
39+
},
40+
})
41+
42+
if err != nil {
43+
log.Fatalf("error calling rpc_service.GetAccountBalance: %s", err)
44+
}
45+
46+
log.Printf("response from remote rpc_service.GetBlockByID(): %s", response)
47+
48+
}

api/handler/accountBalanceHandler.go

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,15 @@ import (
55

66
"github.com/zoobc/zoobc-core/api/service"
77
"github.com/zoobc/zoobc-core/common/model"
8+
"google.golang.org/grpc/codes"
9+
"google.golang.org/grpc/status"
810
)
911

10-
type AccountBalanceHandler struct {
11-
Service service.AccountBalanceServiceInterface
12-
}
12+
type (
13+
AccountBalanceHandler struct {
14+
Service service.AccountBalanceServiceInterface
15+
}
16+
)
1317

1418
func (abh *AccountBalanceHandler) GetAccountBalance(ctx context.Context,
1519
request *model.GetAccountBalanceRequest) (*model.GetAccountBalanceResponse, error) {
@@ -22,6 +26,10 @@ func (abh *AccountBalanceHandler) GetAccountBalance(ctx context.Context,
2226

2327
func (abh *AccountBalanceHandler) GetAccountBalances(ctx context.Context,
2428
request *model.GetAccountBalancesRequest) (*model.GetAccountBalancesResponse, error) {
25-
// todo: implement this after have filter
26-
return nil, nil
29+
30+
if len(request.AccountAddresses) == 0 {
31+
return nil, status.Error(codes.InvalidArgument, "At least 1 address is required")
32+
}
33+
34+
return abh.Service.GetAccountBalances(request)
2735
}
Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
package handler
2+
3+
import (
4+
"context"
5+
"errors"
6+
"reflect"
7+
"testing"
8+
9+
"github.com/zoobc/zoobc-core/api/service"
10+
"github.com/zoobc/zoobc-core/common/model"
11+
)
12+
13+
type (
14+
mockGetAccountBalanceError struct {
15+
service.AccountBalanceServiceInterface
16+
}
17+
mockGetAccountBalanceSuccess struct {
18+
service.AccountBalanceServiceInterface
19+
}
20+
)
21+
22+
func (*mockGetAccountBalanceError) GetAccountBalance(request *model.GetAccountBalanceRequest) (*model.GetAccountBalanceResponse, error) {
23+
return nil, errors.New("error GetAccountBalance")
24+
}
25+
func (*mockGetAccountBalanceSuccess) GetAccountBalance(request *model.GetAccountBalanceRequest) (*model.GetAccountBalanceResponse, error) {
26+
return &model.GetAccountBalanceResponse{
27+
AccountBalance: &model.AccountBalance{
28+
AccountAddress: request.AccountAddress,
29+
},
30+
}, nil
31+
}
32+
33+
func TestAccountBalanceHandler_GetAccountBalance(t *testing.T) {
34+
type fields struct {
35+
Service service.AccountBalanceServiceInterface
36+
}
37+
type args struct {
38+
ctx context.Context
39+
request *model.GetAccountBalanceRequest
40+
}
41+
tests := []struct {
42+
name string
43+
fields fields
44+
args args
45+
want *model.GetAccountBalanceResponse
46+
wantErr bool
47+
}{
48+
{
49+
name: "GetAccountBalance:fail",
50+
fields: fields{
51+
Service: &mockGetAccountBalanceError{},
52+
},
53+
want: nil,
54+
wantErr: true,
55+
},
56+
{
57+
name: "GetAccountBalance:success",
58+
fields: fields{
59+
Service: &mockGetAccountBalanceSuccess{},
60+
},
61+
args: args{
62+
request: &model.GetAccountBalanceRequest{
63+
AccountAddress: "BCZnSfqpP5tqFQlMTYkDeBVFWnbyVK7vLr5ORFpTjgtN",
64+
},
65+
},
66+
want: &model.GetAccountBalanceResponse{
67+
AccountBalance: &model.AccountBalance{
68+
AccountAddress: "BCZnSfqpP5tqFQlMTYkDeBVFWnbyVK7vLr5ORFpTjgtN",
69+
},
70+
},
71+
wantErr: false,
72+
},
73+
}
74+
for _, tt := range tests {
75+
t.Run(tt.name, func(t *testing.T) {
76+
abh := &AccountBalanceHandler{
77+
Service: tt.fields.Service,
78+
}
79+
got, err := abh.GetAccountBalance(tt.args.ctx, tt.args.request)
80+
if (err != nil) != tt.wantErr {
81+
t.Errorf("AccountBalanceHandler.GetAccountBalance() error = %v, wantErr %v", err, tt.wantErr)
82+
return
83+
}
84+
if !reflect.DeepEqual(got, tt.want) {
85+
t.Errorf("AccountBalanceHandler.GetAccountBalance() = %v, want %v", got, tt.want)
86+
}
87+
})
88+
}
89+
}
90+
91+
type (
92+
mockGetAccountBalancesSuccess struct {
93+
service.AccountBalanceServiceInterface
94+
}
95+
)
96+
97+
func (*mockGetAccountBalancesSuccess) GetAccountBalances(request *model.GetAccountBalancesRequest) (*model.GetAccountBalancesResponse, error) {
98+
return &model.GetAccountBalancesResponse{
99+
AccountBalances: []*model.AccountBalance{},
100+
}, nil
101+
}
102+
103+
func TestAccountBalanceHandler_GetAccountBalances(t *testing.T) {
104+
type fields struct {
105+
Service service.AccountBalanceServiceInterface
106+
}
107+
type args struct {
108+
ctx context.Context
109+
request *model.GetAccountBalancesRequest
110+
}
111+
tests := []struct {
112+
name string
113+
fields fields
114+
args args
115+
want *model.GetAccountBalancesResponse
116+
wantErr bool
117+
}{
118+
{
119+
name: "GetAccountBalancesHandler:fail",
120+
args: args{
121+
request: &model.GetAccountBalancesRequest{
122+
AccountAddresses: []string{},
123+
},
124+
},
125+
want: nil,
126+
wantErr: true,
127+
},
128+
{
129+
name: "GetAccountBalancesHandler:success",
130+
args: args{
131+
request: &model.GetAccountBalancesRequest{
132+
AccountAddresses: []string{
133+
"BCZnSfqpP5tqFQlMTYkDeBVFWnbyVK7vLr5ORFpTjgtN",
134+
},
135+
},
136+
},
137+
fields: fields{
138+
Service: &mockGetAccountBalancesSuccess{},
139+
},
140+
want: &model.GetAccountBalancesResponse{
141+
AccountBalances: []*model.AccountBalance{},
142+
},
143+
wantErr: false,
144+
},
145+
}
146+
for _, tt := range tests {
147+
t.Run(tt.name, func(t *testing.T) {
148+
abh := &AccountBalanceHandler{
149+
Service: tt.fields.Service,
150+
}
151+
got, err := abh.GetAccountBalances(tt.args.ctx, tt.args.request)
152+
if (err != nil) != tt.wantErr {
153+
t.Errorf("AccountBalanceHandler.GetAccountBalances() error = %v, wantErr %v", err, tt.wantErr)
154+
return
155+
}
156+
if !reflect.DeepEqual(got, tt.want) {
157+
t.Errorf("AccountBalanceHandler.GetAccountBalances() = %v, want %v", got, tt.want)
158+
}
159+
})
160+
}
161+
}

0 commit comments

Comments
 (0)