Refer Page 1 of this article @https://querydb.blogspot.com/2023/10/hbase-performance-optimization.html
Normally, we run multiple workloads on the cluster. This includes Analytical as well as API calls. This also involves read & write traffic as well...
HBase provides the following mechanisms for managing the performance of a cluster handling multiple workloads: . Quotas . Request Queues . Multiple-Typed Queues
Quotas
HBASE-11598 introduces RPC quotas, which allow you to throttle requests based on the following limits
- Limit overall network throughput and number of RPC requests
- Limit amount of storage used for table or namespaces
- Limit number of tables for each namespace or user
- Limit number of regions for each namespace
For this to work -
- Set the hbase.quota.enabled property in the hbase-site.xml file to true.
- Enter the command to set the set the limit of the quota, type of quota, and to which entity to apply the quota. The command and its syntax are: $hbase_shell> set_quota TYPE =>
For example - set_quota TYPE => THROTTLE, THROTTLE_TYPE => READ, TABLE => 'N1:T1', LIMIT => '2req/sec'
Request Queues
HBASE-10993 introduces such a system for deprioritizing long-running scanners. There are two types of queues, fifo and deadline. To configure the type of queue used, configure the hbase.ipc.server.callqueue.type property in hbase-site.xml. There is no way to estimate how long each request may take, so de-prioritization only affects scans, and is based on the number of “next” calls a scan request has made. An assumption is made that when you are doing a full table scan, your job is not likely to be interactive, so if there are concurrent requests, you can delay long-running scans up to a limit tunable by setting the hbase.ipc.server.queue.max.call.delay property. The slope of the delay is calculated by a simple square root of (numNextCall * weight) where the weight is configurable by setting the hbase.ipc.server.scan.vtime.weight property.
Multiple-Typed Queues
Set following properties -
- hbase.ipc.server.callqueue.handler.factor
- hbase.ipc.server.callqueue.read.ratio
- hbase.ipc.server.callqueue.scan.ratio
For example refer - https://github.com/dinesh028/engineering/raw/3822df5d761b88ffadc4ef580843dfe5128de34f/HBaseUtility/src/main/resources/docs/HBasePropertiesTest.docx
Comments
Post a Comment