Node Log
This page details some important or frequently asked logs from Klaytn nodes. If the Klaytn log is modified or newly added/deleted, please edit this page as well.
For more detailed information about log types, you can refer to log_modules.go
If you encounter any abnormal situation, please report it to the klaytn team via github, Klaytn Forum, or Discord.
Error Logs
Blockchain
CN/PN/EN
########## BAD BLOCK ######### Chain config: %v Number: %v Hash: 0x%x %v Error: %v ##############################
A bad block occurs when the received receipt and the execution result do not match. If a node stops with bad block log, it could be due to two reasons. - Case 1. The configuration of the node is wrong such, as the binary version. - Case 2. There’s a problem with the code. It is very likely that other nodes will also experience the same problem.
This error is critical, so if you see any bad block, please make an issue or report it to the Klaytn GitHub repository.
ConsensusIstanbulCore
CN/PN/EN
Drop an empty message from timeout channel
It means that the round change timer will expire. This error is printed if the timer closes accidentally.
The error may occur when the downloader is started.
check next log is also printed: Block synchronisation started.
NetworksP2P
CN/PN/EN
Protocol istanbul/64 failed id=04680a827fa1b240 conn=staticdial err="write tcp 10.117.2.105:34396->10.117.2.27:32323: use of closed etwork connection" Protocol istanbul/64 failed err="shutting down"
This log can be printed when the other node is disconnected. It is usually followed byDisconnected a P2P Peer log.
Check if the disconnected peer is reconnected again. If it is not reconnected, check the network status or peer connection admin_peers
NodeCN
CN
fail to SendNewBlockHashes err="write tcp 10.117.2.124:24108->10.117.2.108:32323: use of closed network connection" fail to SendNewBlockHashes err="shutting down"
same as Protocol istanbul/64 failed
same as Protocol istanbul/64 failed
NodeCN
CN
fail to SendNewBlock peer=d35220eccdb0de7b err="shutting down"
same as Protocol istanbul/64 failed
same as Protocol istanbul/64 failed
NetworksRPC
EN (mostly)
FastWebsocketHandler fail to upgrade message error="websocket: version != 13"
Version issue of WebSocket connection
The header of the request should contain Sec-Websocket-Version field with the value set at 13. You may not have used klaytn rpc client.
Warn Logs
Blockchain
CN/PN/EN
Upgrade database version from=N/A to=3
It is logged at the beginning of the node start-up
You don't need to handle this.
ConsensusIstanbulCore
CN
[RC] round=
Round change log is started with [RC] tag.
If the round does not end in one or two rounds and continues to go up, then the network status or peer connection should be analyzed first. peer connection check api: admin_peers
ConsensusIstanbulCore
CN
unexpected request address= state="Accept request" seq=312 err="old message" number=311 hash=d960ea…6df6de
A proposer mines a block, but it is turned out unexpected. In most cases, it is too old to be a new block.
You don't need to handle this.
Node
CN/PN/EN
Failed doConnTypeHandshake addr=10.117.2.252:28516 conn=inbound conntype=-1 err="read tcp 10.117.2.78:32324->10.117.2.252:28516: i/o timeout
By dialing, the two P2P peers setup a connection. This log is printed if the setup fails.
Check if the disconnected peer is reconnected again. If not, check the network status or peer connection peer connection check api: admin_peers
NodeCN
PN/EN
Failed to filter bodies peer=c02e4b4d471c56b9 lenTxs=1
A node received the unwanted block header of body when fetching. - lenTxs: non-requested number of txs
You don't need to handle this.
Work
CN
Transaction aborted due to time limit hash=
The block execution time when mining should not exceed 250ms, so the last transaction can be aborted due to this time limit.
Confirm that the transaction enters the block.
Work
CN
Transaction failed, account skipped hash=b1b26c...6b220a err="insufficient balance for transfer" Error(before v1.6.2) Warn(after v1.6.2)
When a transaction cannot be executed during mining due to an insufficient balance in the from account
(Theoretically, it occurs when the balance was sufficient at the time when the transaction was created and entered the txpool, but not at the actual execution time.)
Check if the from account is really out of balance.
Info Logs
Info log contains the additional information for you to know more about the node status, so you don't need to handle Info level log.
Blockchain
CN/PN/EN
Regenerated local transaction journal transactions=0 accounts=0
When node is shut down, local txs are journaled to a file (default file name is transactions.rlp). When node is restarted with the journaled file, the local transactions can be regenerated based on the file. - transactions: number of the regenerated local transaction. - accounts: number of the regenerated accounts(==from)
Blockchain
CN/PN/EN
Inserted a new block number=14 hash=13cbfc…f007fc txs=0 gas=0 elapsed=793.458µs processTxs=167ns finalize=157.708µs validateState=7.542µs totalWrite=443.417µs trieWrite=256.667µs
If the node is not a proposer at that block, and the consensus is successful, the node have executed(==validates) the block. In other words, a block is inserted. - gas: total gas spent during tx execution. This field is commonly used when testing the network to find the used gas per block.
NetworksP2P
CN/PN/EN
[Dial] Add dial candidate from static nodes id=62a08a4b9f091c4b NodeType=0 ip=10.117.2.8 mainPort=32323 port=[32323]
A new P2P peer is connected, and it is a static node. A node added manually by using static-nodes.json or addpeer api is called static node. If it is a multichannel, it uses two ports. ex. [32323, 32324]. - id: dst peer id - NodeType: dst node type(cn,pn,en,bn) - ip: dst ip - mainPort: dst TCP listening port number - port: dst TCP listening port number including both the main port and sub port.
NetworksP2P
CN/PN/EN
Added a multichannel P2P Peer id=28a6760472a078fb conn=staticdial peerID=28a6760472a078fb
A new peer is connected as a multichannel peer. - id/peerID: my node’s peer id - conn: type of connection - inbound: somebody connects me - staticdial: static connection, such as static-nodes.json or addPeer - trusteddial: trusted connection, such as trust-nodes.json. it can be always reconnected and established even the number of connections exceeds max limit.
NetworksP2P
CN/PN/EN
Disconnected a multichannel P2P Peer id=28a6760472a078fb conn=inbound peerID=28a6760472a078fb peerName=Klaytn/v1.7.3+acae89350c/darwin-arm64/go1.18.1 err=EOF
A multichannel peer is disconnected. - peerName: It shows my node info - err: The reason why the connection is disconnected
NetworksP2P
CN/PN/EN
ProtocolManager.processConsensusMsg closed id=28a6760472a078fb conn=inbound PeerName=Klaytn/v1.7.3+acae89350c/darwin-arm64/go1.18.1
When a P2P node is disconnected, the consensus message channel between them is closed, too.
StorageStateDB
CN/PN/EN
Persisted trie from memory database blockNum=23460 updated nodes=4 updated nodes size=499.00B time=539.959µs gcnodes=68 gcsize=10.55kB gctime=226.499µs livenodes=245 livesize=37.80kB
This log is printed to inform you that trie db has been committed. Here, commit means flushing db change to the actual db. A commit is made periodically. - Case 1. If the node is a full node, trie commit is made for every 128 block. - Case 2. If the node is an archive node, trie commit is made for every block. Commit is made at next circumstances, too. - , A commit is made when a node is shut down. - A commit is made when memory size exceeds the cap. Tip. - gc stands for garbage collection. Here, garbage collection means flushing db writes of trie node change. - A full node stores the information of every 128 cycle and the latest 128 blocks. - Archive node stores the information of every block.
Work
CN
Commit new mining work number=14 hash=438ef7…68ca7f txs=0 elapsed=605.375µs commitTime=184.708µs finalizeTime=414.375µs
Every CN mines a block in preparation for round change - number: block number - hash: block hash (it is not a final version) - txs: number of transactions in a block - elapsed: total block mining time (commitTime + finalizeTime) - commitTime: transactions execution time in a block - finalizeTime: block finalize time
Work
CN
Successfully sealed new block number=14 hash=13cbfc…f007fc
[Only Proposer] Sealing a new block is successful. Sealing contains the next steps. - Ibft consensus process for the block. - Update timestamp and signatures of the block
Work
CN
Successfully wrote mined block num=14 hash=13cbfc…f007fc txs=0 elapsed=617.709µs
[Only Proposer] If the node is a proposer, and consensus is succeed, the proposer needs to store the block execution result in the db. This log means the storing was successful.
Work
CN
Mining too far in the future wait=1s
In order to maintain 1 second block creation period, the node sleeps for "1s - previous block generation/propagation/execution time". - wait: how much time the node sleeps
VM
CN/PN/EN
Returning since the addr is not a program account addr=
Somebody tries to call a non-existent contract. Tip. In Klaytn, program account is equivalent to contract account.
Last updated