# Node Log

This page details some important or frequently asked logs from Klaytn nodes. If the Klaytn log is modified or newly added/deleted, please edit this page as well.

For more detailed information about log types, you can refer to [log\_modules.go](https://github.com/klaytn/klaytn/blob/dev/log/log_modules.go)

If you encounter any abnormal situation, please report it to the klaytn team via [github](https://github.com/klaytn/klaytn/issues), [Klaytn Forum](https://forum.klaytn.foundation/), or [Discord](https://discord.io/KlaytnOfficial).

## Error Logs

| Log Type              | Node Type   | Log Message                                                                                                                                                                                                                                            | Description                                                                                                                                                                                                                                                                                                                                                             | Suggested Guide                                                                                                                                                                                                          |
| --------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Blockchain            | CN/PN/EN    | <p>########## BAD BLOCK #########<br>Chain config: %v<br><br>Number: %v<br>Hash: 0x%x<br>%v<br><br>Error: %v<br>##############################</p>                                                                                                     | <p>A bad block occurs when the received receipt and the execution result do not match. If a node stops with bad block log, it could be due to two reasons.<br>- Case 1. The configuration of the node is wrong such, as the binary version.<br>- Case 2. There’s a problem with the code. It is very likely that other nodes will also experience the same problem.</p> | This error is critical, so if you see any bad block, please make an issue or report it to the Klaytn GitHub repository.                                                                                                  |
| ConsensusIstanbulCore | CN/PN/EN    | **Drop an empty message from timeout channel**                                                                                                                                                                                                         | It means that the round change timer will expire. This error is printed if the timer closes accidentally.                                                                                                                                                                                                                                                               | <p>The error may occur when the downloader is started.<br>check next log is also printed: <code>Block synchronisation started</code>.</p>                                                                                |
| NetworksP2P           | CN/PN/EN    | <p><strong>Protocol istanbul/64 failed</strong> id=04680a827fa1b240 conn=staticdial err="write tcp 10.117.2.105:34396->10.117.2.27:32323: use of closed etwork connection"<br><br><strong>Protocol istanbul/64 failed</strong> err="shutting down"</p> | This log can be printed when the other node is disconnected. It is usually followed by`Disconnected a P2P Peer` log.                                                                                                                                                                                                                                                    | Check if the disconnected peer is reconnected again. If it is not reconnected, check the network status or peer connection [admin\_peers](https://docs.klaytn.foundation/dapp/json-rpc/api-references/admin#admin_peers) |
| NodeCN                | CN          | <p><strong>fail to SendNewBlockHashes</strong> err="write tcp 10.117.2.124:24108->10.117.2.108:32323: use of closed network connection"<br><br><strong>fail to SendNewBlockHashes</strong> err="shutting down"</p>                                     | same as `Protocol istanbul/64 failed`                                                                                                                                                                                                                                                                                                                                   | same as `Protocol istanbul/64 failed`                                                                                                                                                                                    |
| NodeCN                | CN          | **fail to SendNewBlock** peer=d35220eccdb0de7b err="shutting down"                                                                                                                                                                                     | same as `Protocol istanbul/64 failed`                                                                                                                                                                                                                                                                                                                                   | same as `Protocol istanbul/64 failed`                                                                                                                                                                                    |
| NetworksRPC           | EN (mostly) | **FastWebsocketHandler fail to upgrade message** error="websocket: version != 13"                                                                                                                                                                      | Version issue of WebSocket connection                                                                                                                                                                                                                                                                                                                                   | The header of the request should contain `Sec-Websocket-Version` field with the value set at 13. You may not have used klaytn rpc client.                                                                                |

## Warn Logs

| Log Type              | Node Type | Log Message                                                                                                                                                                | Description                                                                                                                                                                                                                                                                                         | Suggested Guide                                                                                                                                                                                                                                                                               |
| --------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Blockchain            | CN/PN/EN  | **Upgrade database version** from=N/A to=3                                                                                                                                 | It is logged at the beginning of the node start-up                                                                                                                                                                                                                                                  | You don't need to handle this.                                                                                                                                                                                                                                                                |
| ConsensusIstanbulCore | CN        | **\[RC]** round=                                                                                                                                                           | Round change log is started with \[RC] tag.                                                                                                                                                                                                                                                         | <p>If the round does not end in one or two rounds and continues to go up, then the network status or peer connection should be analyzed first.<br><br>peer connection check api: <a href="https://docs.klaytn.foundation/dapp/json-rpc/api-references/admin#admin_peers">admin\_peers</a></p> |
| ConsensusIstanbulCore | CN        | **unexpected request** address= state="Accept request" seq=312 err="old message" number=311 hash=d960ea…6df6de                                                             | A proposer mines a block, but it is turned out unexpected. In most cases, it is too old to be a new block.                                                                                                                                                                                          | You don't need to handle this.                                                                                                                                                                                                                                                                |
| Node                  | CN/PN/EN  | **Failed doConnTypeHandshake** addr=10.117.2.252:28516 conn=inbound conntype=-1 err="read tcp 10.117.2.78:32324->10.117.2.252:28516: i/o timeout                           | By dialing, the two P2P peers setup a connection. This log is printed if the setup fails.                                                                                                                                                                                                           | <p>Check if the disconnected peer is reconnected again. If not, check the network status or peer connection<br><br>peer connection check api: <a href="https://docs.klaytn.foundation/dapp/json-rpc/api-references/admin#admin_peers">admin\_peers</a></p>                                    |
| NodeCN                | PN/EN     | **Failed to filter bodies** peer=c02e4b4d471c56b9 lenTxs=1                                                                                                                 | <p>A node received the unwanted block header of body when fetching.<br>- lenTxs: non-requested number of txs</p>                                                                                                                                                                                    | You don't need to handle this.                                                                                                                                                                                                                                                                |
| Work                  | CN        | **Transaction aborted due to time limit** hash=                                                                                                                            | The block execution time when mining should not exceed 250ms, so the last transaction can be aborted due to this time limit.                                                                                                                                                                        | Confirm that the transaction enters the block.                                                                                                                                                                                                                                                |
| Work                  | CN        | <p><strong>Transaction failed, account skipped</strong> hash=b1b26c...6b220a err="insufficient balance for transfer"<br><br>Error(before v1.6.2)<br>Warn(after v1.6.2)</p> | <p>When a transaction cannot be executed during mining due to an insufficient balance in the <code>from</code> account<br>(Theoretically, it occurs when the balance was sufficient at the time when the transaction was created and entered the txpool, but not at the actual execution time.)</p> | Check if the `from` account is really out of balance.                                                                                                                                                                                                                                         |

## Info Logs

`Info` log contains the additional information for you to know more about the node status, so you don't need to handle `Info` level log.

| Log Type       | Node Type | Log Message                                                                                                                                                                                | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| -------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Blockchain     | CN/PN/EN  | **Regenerated local transaction journal** transactions=0 accounts=0                                                                                                                        | <p>When node is shut down, local txs are journaled to a file (default file name is transactions.rlp). When node is restarted with the journaled file, the local transactions can be regenerated based on the file.<br>- transactions: number of the regenerated local transaction.<br>- accounts: number of the regenerated accounts(==from)</p>                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Blockchain     | CN/PN/EN  | **Inserted a new block** number=14 hash=13cbfc…f007fc txs=0 gas=0 elapsed=793.458µs processTxs=167ns finalize=157.708µs validateState=7.542µs totalWrite=443.417µs trieWrite=256.667µs     | <p>If the node is not a proposer at that block, and the consensus is successful, the node have executed(==validates) the block. In other words, a block is inserted.<br>- gas: total gas spent during tx execution.<br>This field is commonly used when testing the network to find the used gas per block.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| NetworksP2P    | CN/PN/EN  | **\[Dial] Add dial candidate from static nodes** id=62a08a4b9f091c4b NodeType=0 ip=10.117.2.8 mainPort=32323 port=\[32323]                                                                 | <p>A new P2P peer is connected, and it is a static node. A node added manually by using static-nodes.json or addpeer api is called static node. If it is a multichannel, it uses two ports. ex. \[32323, 32324].<br>- id: dst peer id<br>- NodeType: dst node type(cn,pn,en,bn)<br>- ip: dst ip<br>- mainPort: dst TCP listening port number<br>- port: dst TCP listening port number including both the main port and sub port.</p>                                                                                                                                                                                                                                                                                                                                             |
| NetworksP2P    | CN/PN/EN  | **Added a multichannel P2P Peer** id=28a6760472a078fb conn=staticdial peerID=28a6760472a078fb                                                                                              | <p>A new peer is connected as a multichannel peer.<br>- id/peerID: my node’s peer id<br>- conn: type of connection<br>- inbound: somebody connects me<br>- staticdial: static connection, such as static-nodes.json or addPeer<br>- trusteddial: trusted connection, such as trust-nodes.json. it can be always reconnected and established even the number of connections exceeds max limit.</p>                                                                                                                                                                                                                                                                                                                                                                                |
| NetworksP2P    | CN/PN/EN  | **Disconnected a multichannel P2P Peer** id=28a6760472a078fb conn=inbound peerID=28a6760472a078fb peerName=Klaytn/v1.7.3+acae89350c/darwin-arm64/go1.18.1 err=EOF                          | <p>A multichannel peer is disconnected.<br>- peerName: It shows my node info<br>- err: The reason why the connection is disconnected</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| NetworksP2P    | CN/PN/EN  | **ProtocolManager.processConsensusMsg closed** id=28a6760472a078fb conn=inbound PeerName=Klaytn/v1.7.3+acae89350c/darwin-arm64/go1.18.1                                                    | When a P2P node is disconnected, the consensus message channel between them is closed, too.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| StorageStateDB | CN/PN/EN  | **Persisted trie from memory database** blockNum=23460 updated nodes=4 updated nodes size=499.00B time=539.959µs gcnodes=68 gcsize=10.55kB gctime=226.499µs livenodes=245 livesize=37.80kB | <p>This log is printed to inform you that trie db has been committed. Here, commit means flushing db change to the actual db.<br><br>A commit is made periodically.<br>- Case 1. If the node is a full node, trie commit is made for every 128 block.<br>- Case 2. If the node is an archive node, trie commit is made for every block.<br><br>Commit is made at next circumstances, too.<br>- , A commit is made when a node is shut down.<br>- A commit is made when memory size exceeds the cap.<br><br>Tip.<br>- gc stands for garbage collection. Here, garbage collection means flushing db writes of trie node change.<br>- A full node stores the information of every 128 cycle and the latest 128 blocks.<br>- Archive node stores the information of every block.</p> |
| Work           | CN        | **Commit new mining work** number=14 hash=438ef7…68ca7f txs=0 elapsed=605.375µs commitTime=184.708µs finalizeTime=414.375µs                                                                | <p>Every CN mines a block in preparation for round change<br>- number: block number<br>- hash: block hash (it is not a final version)<br>- txs: number of transactions in a block<br>- elapsed: total block mining time (commitTime + finalizeTime)<br>- commitTime: transactions execution time in a block<br>- finalizeTime: block finalize time</p>                                                                                                                                                                                                                                                                                                                                                                                                                           |
| Work           | CN        | **Successfully sealed new block** number=14 hash=13cbfc…f007fc                                                                                                                             | <p>\[Only Proposer] Sealing a new block is successful. Sealing contains the next steps.<br>- Ibft consensus process for the block.<br>- Update timestamp and signatures of the block</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| Work           | CN        | **Successfully wrote mined block** num=14 hash=13cbfc…f007fc txs=0 elapsed=617.709µs                                                                                                       | \[Only Proposer] If the node is a proposer, and consensus is succeed, the proposer needs to store the block execution result in the db. This log means the storing was successful.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| Work           | CN        | **Mining too far in the future** wait=1s                                                                                                                                                   | <p>In order to maintain 1 second block creation period, the node sleeps for "1s - previous block generation/propagation/execution time".<br>- wait: how much time the node sleeps</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| VM             | CN/PN/EN  | **Returning since the addr is not a program account** addr=                                                                                                                                | <p>Somebody tries to call a non-existent contract.<br><br>Tip. In Klaytn, program account is equivalent to contract account.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
