Tiered Storage

tip

Only Enterprise Edition supports

In the actual service scenario, there are multiple data disks on one machine at the same time. Therefore, it is necessary to give full play to the capabilities of these disks to improve the throughput of the system. Another scenario is that multiple data disks on the same or multiple machines use different storage media, and customers want to store cold data that is not frequently accessed move to cheap storage media to reduce the cost of enterprise storage. In CnosDB, we solve these two scenarios through the following solutions: In CnosDB, we solve these two scenarios by the following solutions:

Tiered Storage

The basic unit of data migration in a CnosDB cluster is the Vnode.The enterprise version of CnosDB cluster provides the following commands to support hierarchical storage:

SHOW DATANODES;                          // Show all nodes in the cluster
+---------+------------------------+-----------+---------+-----------+---------------------+
| NODE_ID | HOST                   | ATTRIBUTE | STATUS  | DISK_FREE | LAST_UPDATED_TIME   |
+---------+------------------------+-----------+---------+-----------+---------------------+
| 1001    | query_tskv1.cnosdb.com | HOT       | HEALTHY | 5.18 GB   | 2023-06-05 02:30:22 |
| 1002    | query_tskv2.cnosdb.com | HOT       | HEALTHY | 93.71 GB  | 2023-06-05 02:30:19 |
+---------+------------------------+-----------+---------+-----------+---------------------+

ALTER NODE [node_id] ATTRIBUTE [HOT/COLD];  // Modify the attribute of the node

CREATE DATABASE [IF NOT EXISTS] db_name [WITH db_options];

db_options:
    db_option ...

db_option: {
      TTL value
    | SHARD value
    | VNODE_DURATION value
    | REPLICA value
    | COOLING_DURATION value
    | PRECISION {'ms' | 'us' | 'ns'}
}

Add a COOLING_DURATION value to the CREATE DATABASE statement to add a cooling time field. Add COOLING_DURATION value to the CREATE DATABASE statement to add a cooldown time field. option indicates the interval for data cooling, COOLING_DURATION defaults to 0, which means infinite length.

COOLING_DURATION must be an integer multiple of VNODE_DURATION. COOLING_DURATION must be a multiple of VNODE_DURATION. The COOLING_DURATION field can be modified by alter database.

After data is cooled, the migrate thread in meta will migrate data from hot nodes to cold nodes. The migrate thread periodically checks whether there is data to be migrated. The timing can be modified using the auto_migrate_vnodes_duration configuration item in the meta configuration file. The unit of this configuration item is seconds, and the default value is 0, which means that tiered storage is not enabled. The timing for periodic checks can be set according to practical needs. It is recommended to set the minimum check time to 1800 seconds, which is auto_migrate_vnodes_duration = 1800.

Note

• After data is cooled, data is migrated from the hot node to the cold node. However, the cooling degree of data is defined by users. Users can modify the cooling period of db to change the cooling degree of data.

• Users change the db cool downtime. The change may migrate data from a cold node to a hot node, for example: When starting the cluster through ./run_cluster.sh with default configuration, there are two data nodes in the cluster: 1001 and 2001, and both are hot nodes by default.

ALTER NODE 2001 ATTRIBUTE COLD;  // 修改data节点2001为冷节点
CREATE DATABASE db1 with VNODE_DURATION '1m' COOLING_DURATION '1m';  // 在1001节点上创建一个名为db1的数据库，它的VNODE_DURATION和COOLING_DURATION都设置为1分钟

Then create a table and write data in the "db1" database, wait for a period of time until the data is cooled and migrated from hot node 1001 to cold node 2001. At this point, execute the SQL statement:

ALTER DATABASE db1 SET COOLING_DURATION '1d'; // 将数据库db1的冷却时间由上面的1分钟，改为1天

At this point, as the cool downtime becomes larger, the cooled data above becomes hot data again, so the data is moved from cold node 2001 to hot node 1001.