Hadoop HBase 碎碎念

Fri, 25 Mar 2016 22:08:12 +0800

一些小笔记会持续追加到这里

2016-3-25 22:09:24

重启或添加节点： $bin/hadoop-daemon.sh start datanode

-————————-

2016-3-25 22:09:31

Had the same problem with 2.6.0, and shamouda’s answer solved it (I was not using dfs.hosts at all so that could not be the answer. I did add

<property>
  <name>dfs.namenode.datanode.registration.ip-hostname-check</name>
  <value>false</value>
</property>

to hdfs-site.xml and that was enough to fix the issue.

-———————-

2016-3-25 22:11:55

$bin/hbase-daemon.sh stop regionserver $bin/hbase-daemon.sh start regionserver

-———————–

2016-3-27 16:54:22

disable ‘kline_minute’ describe ‘kline_minute’ alter ‘kline_minute’ , NAME=>‘kline’, COMPRESSION=>‘gz’ describe ‘kline_minute’ enable ‘kline_minute’ major_compact ‘kline_minute’

启用压缩

http://www.searchtb.com/2011/01/understanding-hbase.html，这里说明了文件的存储方式 http://stackoverflow.com/questions/18656483/data-size-increases-in-hbase，这里有一个计算的例子

-———————–

2016-3-27 16:56:14

重启一个regionserver

bin/graceful_stop.sh –restart –reload –debugnodename

这个操作是平滑的重启regionserver进程，对服务不会有影响，他会先将需要重启的regionserver上面的所有region迁移到其它的服务器，然后重启，最后又会将之前的region迁移回来，但我们修改一个配置时，可以用这种方式重启每一台机子，这个命令会关闭balancer，所以最后我们要在hbase shell里面执行一下balance_switch true，对于hbase regionserver重启，不要直接kill进程，这样会造成在zookeeper.session.timeout这个时间长的中断，也不要通过bin/hbase-daemon.sh stop regionserver去重启，如果运气不太好，-ROOT-或者.META.表在上面的话，所有的请求会全部失败。

关闭下线一台regionserver

bin/graceful_stop.sh –stop nodename

和上面一样，系统会在关闭之前迁移所有region，然后stop进程，同样最后我们要手工balance_switch true，开启master的region均衡。

hbase 创建表时提示存在删除时时又提示不存在

Mon, 21 Mar 2016 12:20:02 +0800

看到这里，http://arnon.me/2013/01/killing-hbase-zombie-table/

One of our team leaders approached me in the hall today and asked if I could land a hand in troubleshooting something. He and our QA lead were configuring one of our test Hadoop clusters after an upgrade and they had a problem with one table they were trying to set up:

When they tried to create the table in HBase shell they got an error that the table exists When they tried to delete the table they got an error that the table does not exist HBase ships with a health-check and fix util called hbck (use: hbase hbck to run. see here for details) – they’ve run hbase reports everything is fine and dandy Hmm, The first thing I tied to do is to look at the .META. table. This is where HBase keeps the tables and the regions they use. I thought maybe there was some just there. but it didn’t look like that. I tried to do a major compaction for it and that didn’t help either.

The next thing I tried actually found the problem. I ran the Zookeeper client (I used hbase zkcli but you can also run it via zookeeper scripts) and looked at /hbase/table (ls /hbase table) -the zombie table was listed right there with all the legit tables. HBase stores some data schema and state of each table in zookeeper to be able to coordinate between all the regionservers and it seems that during the upgrade process the system was restarted a few times. One of these restarts coincided with a removal of the table and caught it in the middle.

Ok, so that is the problem – what’s the solution? Simple just remove the offending znode from zookeeper (rmr /hbase/table/TABLE_NAME ) and restart the cluster (since the data is cached in the regionservers/hbase master to save trips to zookeeper). Also be careful not to remove any other node or you’d cause problems to other tables.

The role of ZooKeeper in HBase is not documented very well. The only online account of ZooKeeper’s role with HBase I found (save looking at the code itself of course) is really outdated. Hopefully this post will save some head scratching and time for others who find themselves with the same problem.

Anyway, I hope the next post I’ll do on ZooKeeper will be about something much nicer :)

嗯，hbase 自带了一个 zkcli，所以操作起来还是比较简单的

Hbase on ZRJ | 学习笔记

Hadoop HBase 碎碎念

hbase 创建表时提示存在删除时时又提示不存在