博客
关于我
强烈建议你试试无所不能的chatGPT，快点击我
namenode的log时，散仙发现有如下的警告信息
阅读量：4188 次
发布时间：2019-05-26

本文共 12655 字，大约阅读时间需要 42 分钟。

刚装完的hadoop集群，在查看namenode的log时，发现有如下的警告信息：

Java代码  
  1. 2014-10-30 16:49:18,340 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 2 secs  
  2. 2014-10-30 16:49:18,340 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes  
  3. 2014-10-30 16:49:18,340 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks  
  4. 2014-10-30 16:49:18,485 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode RPC up at: fsedump00yw/192.168.223.163:8020  
  5. 2014-10-30 16:49:18,485 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state  
  6. 2014-10-30 16:49:18,452 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting  
  7. 2014-10-30 16:49:18,454 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8020: starting  
  8. 2014-10-30 16:49:25,377 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(192.168.223.168, storageID=DS-1779348018-192.168.223.168-50010-1414658969808, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0) storage DS-1779348018-192.168.223.168-50010-1414658969808  
  9. 2014-10-30 16:49:25,392 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.223.168:50010  
  10. 2014-10-30 16:49:25,983 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 192.168.223.168:50010 after starting up or becoming active. Its block contents are no longer considered stale  
  11. 2014-10-30 16:49:25,984 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(192.168.223.168, storageID=DS-1779348018-192.168.223.168-50010-1414658969808, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0), blocks: 0, processing time: 15 msecs  
  12. 2014-10-30 16:49:25,998 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(192.168.223.163, storageID=DS-335820978-192.168.223.163-50010-1414658964818, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0) storage DS-335820978-192.168.223.163-50010-1414658964818  
  13. 2014-10-30 16:49:26,002 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.223.163:50010  
  14. 2014-10-30 16:49:26,409 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 192.168.223.163:50010 after starting up or becoming active. Its block contents are no longer considered stale  
  15. 2014-10-30 16:49:26,409 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(192.168.223.163, storageID=DS-335820978-192.168.223.163-50010-1414658964818, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0), blocks: 0, processing time: 0 msecs  
  16. 2014-10-30 16:49:31,077 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(192.168.223.167, storageID=DS-563266444-192.168.223.167-50010-1414682451971, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0) storage DS-563266444-192.168.223.167-50010-1414682451971  
  17. 2014-10-30 16:49:31,078 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.223.167:50010  
  18. 2014-10-30 16:49:31,174 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 192.168.223.167:50010 after starting up or becoming active. Its block contents are no longer considered stale  
  19. 2014-10-30 16:49:31,174 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(192.168.223.167, storageID=DS-563266444-192.168.223.167-50010-1414682451971, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0), blocks: 0, processing time: 0 msecs  
  20. 2014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 192.168.223.163  
  21. 2014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs  
  22. 2014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 1  
  23. 2014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 30   
  24. 2014-10-30 16:50:31,873 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 30   
  25. 2014-10-30 16:50:31,876 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /data/hadoop-nd/current/edits_inprogress_0000000000000000001 -> /data/hadoop-nd/current/edits_0000000000000000001-0000000000000000002  
  26. 2014-10-30 16:50:31,876 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 3  
  27. 2014-10-30 16:50:33,246 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection to http://fsedump00yw:8021/getimage?getimage=1&txid=2&storageInfo=-47:1726826048:0:CID-0f002fcf-1b19-4004-a76e-a736b0b98df1  
  28. 2014-10-30 16:50:33,876 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Transfer took 0.63s at 0.00 KB/s  
  29. 2014-10-30 16:50:33,876 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000002 size 198 bytes.  
  30. 2014-10-30 16:50:33,888 INFO org.apache.hadoop.hdfs.server.namenode.NNStorageRetentionManager: Going to retain 2 images with txid >= 0  
  31. 2014-10-30 16:53:22,379 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: trying to get DT with no secret manager running  
  32. 2014-10-30 16:53:23,180 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user dr.who  
  33. 2014-10-30 16:53:23,253 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user dr.who  
2014-10-30 16:49:18,340 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 2 secs2014-10-30 16:49:18,340 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes2014-10-30 16:49:18,340 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks2014-10-30 16:49:18,485 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode RPC up at: fsedump00yw/192.168.223.163:80202014-10-30 16:49:18,485 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state2014-10-30 16:49:18,452 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting2014-10-30 16:49:18,454 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8020: starting2014-10-30 16:49:25,377 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(192.168.223.168, storageID=DS-1779348018-192.168.223.168-50010-1414658969808, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0) storage DS-1779348018-192.168.223.168-50010-14146589698082014-10-30 16:49:25,392 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.223.168:500102014-10-30 16:49:25,983 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 192.168.223.168:50010 after starting up or becoming active. Its block contents are no longer considered stale2014-10-30 16:49:25,984 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(192.168.223.168, storageID=DS-1779348018-192.168.223.168-50010-1414658969808, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0), blocks: 0, processing time: 15 msecs2014-10-30 16:49:25,998 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(192.168.223.163, storageID=DS-335820978-192.168.223.163-50010-1414658964818, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0) storage DS-335820978-192.168.223.163-50010-14146589648182014-10-30 16:49:26,002 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.223.163:500102014-10-30 16:49:26,409 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 192.168.223.163:50010 after starting up or becoming active. Its block contents are no longer considered stale2014-10-30 16:49:26,409 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(192.168.223.163, storageID=DS-335820978-192.168.223.163-50010-1414658964818, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0), blocks: 0, processing time: 0 msecs2014-10-30 16:49:31,077 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(192.168.223.167, storageID=DS-563266444-192.168.223.167-50010-1414682451971, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0) storage DS-563266444-192.168.223.167-50010-14146824519712014-10-30 16:49:31,078 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.223.167:500102014-10-30 16:49:31,174 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: BLOCK* processReport: Received first block report from 192.168.223.167:50010 after starting up or becoming active. Its block contents are no longer considered stale2014-10-30 16:49:31,174 INFO BlockStateChange: BLOCK* processReport: from DatanodeRegistration(192.168.223.167, storageID=DS-563266444-192.168.223.167-50010-1414682451971, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-0f002fcf-1b19-4004-a76e-a736b0b98df1;nsid=1726826048;c=0), blocks: 0, processing time: 0 msecs2014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 192.168.223.1632014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs2014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 12014-10-30 16:50:31,872 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 30 2014-10-30 16:50:31,873 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 30 2014-10-30 16:50:31,876 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /data/hadoop-nd/current/edits_inprogress_0000000000000000001 -> /data/hadoop-nd/current/edits_0000000000000000001-00000000000000000022014-10-30 16:50:31,876 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 32014-10-30 16:50:33,246 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection to http://fsedump00yw:8021/getimage?getimage=1&txid=2&storageInfo=-47:1726826048:0:CID-0f002fcf-1b19-4004-a76e-a736b0b98df12014-10-30 16:50:33,876 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Transfer took 0.63s at 0.00 KB/s2014-10-30 16:50:33,876 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000002 size 198 bytes.2014-10-30 16:50:33,888 INFO org.apache.hadoop.hdfs.server.namenode.NNStorageRetentionManager: Going to retain 2 images with txid >= 02014-10-30 16:53:22,379 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: trying to get DT with no secret manager running2014-10-30 16:53:23,180 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user dr.who2014-10-30 16:53:23,253 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user dr.who

注意上述代码的最后几行发出的警告信息，意思是说对于dr.who这个用户，没有有效的组归属，在hadoop的权限认证中，如果默认没有用户组声明，就会把当前的这个hadoop用户，归属于dr.who用户，当然这个用户在我们的用户组里面是不存在的，所以log里面就记录了这个warn信息。
解决方法如下：
方法（1），在core-site.xml里面添加如下配置：

Java代码  
  1. <property>  
  2.     <name>hadoop.http.staticuser.user</name>  
  3.     <value>hadoop</value>  
  4. </property>  
        
     
      hadoop.http.staticuser.user
         
     
      hadoop

hadoop是散仙的用户名
方法（2），在当前的用户组里面添加一个dr.who的用户
gpasswd -a dr.who hadoop
将dr.who用户添加到hadoop用户组里面
如此就能通过linux的权限验证，改完之后，重启hadoop集群之后，就不会出现这样的警告了

转载地址：http://cxjoi.baihongyu.com/

你可能感兴趣的文章
买卖股票的最佳时机 II
查看>>
分发饼干
查看>>
最低票价
查看>>
删列造序
查看>>
使括号有效的最少添加
查看>>
令牌放置
查看>>
回溯法思想
查看>>
子集和问题
查看>>
旅行售货员问题
查看>>
区域和检索 - 数组不可变
查看>>
整数分解
查看>>
最长有效括号
查看>>
救生艇
查看>>
Android中自定义圆形图片（一）
查看>>
Android中ViewPager自动加手动轮播
查看>>
Android中Fragment点击切换与添加ViewPager滑动切换
查看>>
二级MSOffice高级应用考试大纲(2013年版)
查看>>
图解LogCat的用法
查看>>
学习Android界面设计的超级利器HierarchyView.bat
查看>>
众多Android 开源项目推荐，给力工作给力学习
查看>>
喝酒易醉，品茶养心，人生如梦，品茶悟道，何以解忧？唯有杜康！-- 愿君每日到此一游！
当前时间: 2024-09-24 16:29:50   当前IP: 18.191.176.228   联系邮箱:javaeecc@qq.com     Copyright © 2020 - 2022 baihongyu.com 京ICP备2021015314号-2
强烈建议你试试无所不能的CHAT-GPT，快点击我
强烈建议你试试无所不能的CHAT-GPT，快点击我