Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

taosdump 3.2.3.0 import stuck while using multi threads #26297

Open
Whitroom opened this issue Jun 26, 2024 · 0 comments
Open

taosdump 3.2.3.0 import stuck while using multi threads #26297

Whitroom opened this issue Jun 26, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@Whitroom
Copy link

Bug描述
TDEngine3.2.3.0对数据进行taosdump导出后，删除 /var/lib/taos/** 重启taosd后使用taosdump导入卡住不动

如何重现
使用systemctl进行进程管理

taosdump -uroot --database=ess_cloud -o td-bak
systemctl stop taosd
systemctl stop taosadapter
rm -rf /var/lib/taos/**
systemctl start taosd
systemctl start taosadapter
# 创建demo数据库
nohup taosdump -uroot --database=ess_cloud -i td-bak -g &

期待结果
可以成功导入

日志

DEBG: Client info: 3.2.3.0, major version: 3
==============================
========== arguments config =========
version: 3.2.3.0
gitinfo: 3bb61cc
buildInfo: Built Linux-x64 at 2024-02-29 18:01:09 +0800
 host: (null)
user: root
port: 0
outpath: 
inpath: td-bak
resultFile: ./dump_result.txt
all_databases: false
databases: true
databasesSeq: ess_cloud
schemaonly: false
with_property: true
avro codec: snappy
data_batch: 16383
thread_num: 128
allow_sys: false
escape_char: true
loose_mode: false
isDumpIn: true
arg_list_len: 0
buildInfo: Built Linux-x64 at 2024-02-29 18:01:09 +0800
 start time: 2024-06-26 11:34:09
INFO: dumpInWithDbPath(), dump in from td-bak ...
DEBG: Success Open input file: td-bak/dbs.sql
DEBG: dumpInDbs() LN10525, dump in data minor version is: 2
INFO: dumpInAvroWorkThreads() dump in avro-tbtags files ...
DEBG: 0 .avro-tbtags files found!
DEBG: No .avro-tbtags file found in td-bak
INFO: dumpInAvroWorkThreads() dump in avro-ntb files ...
DEBG: 0 .avro-ntb files found!
DEBG: No .avro-ntb file found in td-bak
INFO: dumpInAvroWorkThreads() dump in avro files ...
DEBG: 0 .avro files found!
DEBG: No .avro file found in td-bak
DEBG: dumpIn() LN10643, will dump from td-bak/taosdump.3438732572160
NFO: dumpInWithDbPath(), dump in from td-bak/taosdump.3438732572160 ...
DEBG: Success Open input file: td-bak/taosdump.3438732572160/dbs.sql
DEBG: dumpInDbs() LN10525, dump in data minor version is: 2
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE DATABASE IF NOT EXISTS ess_cloud REPLICA 1   DURATION 14400m KEEP 5256000m,5256000m,5256000m     PRECISION 'ms'   MINROWS 100 MAXROWS 4096 COMP 2 ;
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`aircool_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`fault_state` 
bigint,`work_state` tinyint,`indoor_fan_state` tinyint,`outdoor_fan_state` tinyint,`compr_state` tinyint,`heat_state` tinyint,`defrost_temp` float,`condense_temp` float,`outdoor_temp` float,`indoor_temp` float,`indo
or_humi` float,`air_out_temp` float,`cool_temp` float,`cool_dif` float,`heat_temp` float,`heat_dif` float,`high_temp` float,`low_temp` float,`ctrl_humi` float,`humi_dif` float,`high_humi` float) TAGS(`air_code` bina
ry(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`liquidcool_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`run_state`
 tinyint,`compr_state` tinyint,`heat_state` tinyint,`self_loop_state` tinyint,`pump_speed` float,`outdoor_temp` float,`inlet_water_temp` float,`outlet_water_temp` float,`inlet_water_press` float,`outlet_water_press`
 float,`fan_output` float,`fault_level` tinyint,`state_info` binary(64),`fault_info` binary(64),`on_off` tinyint,`set_run_mode` tinyint,`set_temp` float,`set_diff_mode` tinyint,`cool_start_dif` float,`cool_stop_dif`
 float,`heat_start_dif` float,`heat_stop_dif` float) TAGS(`liquid_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`event_data`(`ts` timestamp,`event_level` nchar(16),`senior_type` nchar(16),`junior_type` nchar(16),`event_desc` nchar(256),`collect_time
` nchar(32),`write_time` nchar(32),`event_id` int,`device_type` nchar(32),`device_manufactor` nchar(32),`device_id` nchar(128),`event_status` int,`event_data_detail` nchar(256),`box_code` nchar(16),`bms_code` nchar(
16)) TAGS(`station_code` nchar(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`device_pcs_overview`(`ts` timestamp,`work_state` tinyint) TAGS(`pcs_code` binary(32),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`fenggu_data`(`ts` timestamp,`down_id` nchar(32),`down_time` nchar(32),`down_status` int,`down_user` nchar(32),`down_user_name` nchar(32)
,`down_user_phone` nchar(32),`down_data` nchar(4096)) TAGS(`station_code` nchar(32));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`fire_host_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`alarm_state
` tinyint,`fault_state` tinyint,`spary_state` tinyint) TAGS(`fire_host_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`cluster_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`soc` float,`soh` float,`voltage` double,`current` float,`le
ft_energy` double,`used_energy` double,`input_ah` double,`output_ah` double,`warn_state` bigint,`prt_state` bigint,`input_watt` double,`output_watt` double,`day_input_watt` double,`day_output_watt` double,`vmin` flo
at,`vmin_bmu_idx` smallint,`vmin_cell_idx` smallint,`vmax` float,`vmax_bmu_idx` smallint,`vmax_cell_idx` smallint,`tmin` float,`tmin_bmu_idx` smallint,`tmin_cell_idx` smallint,`tmax` float,`tmax_bmu_idx` smallint,`t
max_cell_idx` smallint,`vmin_sec` float,`vmin_sec_bmu_idx` smallint,`vmin_sec_cell_idx` smallint,`vmax_sec` float,`vmax_sec_bmu_idx` smallint,`vmax_sec_cell_idx` smallint,`tmin_sec` float,`tmin_sec_bmu_idx` smallint
,`tmin_sec_cell_idx` smallint,`tmax_sec` float,`tmax_sec_bmu_idx` smallint,`tmax_sec_cell_idx` smallint,`bmu_online_state` bigint,`run_state` bigint,`bc_prt_act_state` bigint,`brk_fault_state` bigint,`sa_online_stat
e` tinyint,`bc_self_chk_state` tinyint,`bc_fault_state` tinyint,`balance_state` bigint,`balance_en` bigint,`brk_state` bigint,`bc_sys_state` tinyint,`bmu_volt_sum` double,`power` double,`bcms_online_state` bigint,`b
cu_online_state` bigint,`main_breaker_state` tinyint,`balance_mode` tinyint,`balance_manual_mode_en` tinyint,`fan_state` int,`fan_fault` int,`limit_power_state` bigint,`limit_chg_power` double,`limit_dchg_power` dou
ble,`bmu_avg_vol` float,`cell_avg_vol` float,`cell_avg_temp` float,`vol_dif_max` float,`temp_dif_max` float,`soe` float,`sop` float,`sof` float,`pbus_resistor` float,`nbus_resistor` float,`correct` tinyint) TAGS(`cl
uster_code` binary(32),`bms_code` binary(16),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`dehumidity_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`env_temp` 
float,`cool_temp` float,`env_humi` float,`ctrl_temp` float,`ctrl_humi` float,`humi_dif` float,`fan_state` tinyint,`cool_state` tinyint,`heat_state` tinyint,`run_mode` tinyint,`fan_fault` tinyint,`cool_fault` tinyint
,`sensor_fault` tinyint,`over_humi_alarm` tinyint,`over_temp_alarm` tinyint) TAGS(`dehumidity_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`station_overview`(`ts` timestamp,`write_time` timestamp,`upper_limit_soc` float,`lower_limit_soc` float,`total_power` double,`dischargab
le_energy` double,`chargable_energy` double,`today_discharged_energy` double,`today_charged_energy` double,`max_temp` float,`min_temp` float,`max_soc` float,`min_soc` float,`station_soc` float,`poc_power` double,`po
c_on_grid_energy` double,`poc_off_grid_energy` double,`ess_power` double,`load_power` double,`discharged_energy` double,`charged_energy` double,`rated_capacity` double,`rated_power` double,`max_temp_location` binary
(32),`min_temp_location` binary(32),`max_soc_location` binary(32),`min_soc_location` binary(32),`device_type` binary(16),`correct` tinyint) TAGS(`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`leak_detector_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`leak_st
ate` int) TAGS(`leak_detector_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`pcs_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`online_state` tinyint,`ua` double,`ub` double,`uc` double,`ia` 
double,`ib` double,`ic` double,`grid_freq` float,`active_power` double,`reactive_power` double,`apparent_power` double,`power_factor` double,`udc` double,`idc` double,`pdc` double,`igbt` float,`igbta` float,`igbtb` 
float,`igbtc` float,`expect_active_power` double,`expect_reactive_power` double,`work_state` tinyint,`state_info` binary(64),`fault_info` binary(64),`ctrl_active_power` double,`ctrl_reactive_power` double,`ctrl_on_o
ff` tinyint,`status` tinyint) TAGS(`pcs_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`meter_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`online_state` tinyint,`ua` double,`ub` double,`uc` double,`ia
` double,`ib` double,`ic` double,`grid_freq` float,`active_power` double,`reactive_power` double,`apparent_power` double,`power_factor` double,`combined_active_total` double,`combined_active_tip` double,`combined_ac
tive_peak` double,`combined_active_flat` double,`combined_active_valley` double,`positive_active_total` double,`positive_active_tip` double,`positive_active_peak` double,`positive_active_flat` double,`positive_activ
e_valley` double,`reverse_active_total` double,`reverse_active_tip` double,`reverse_active_peak` double,`reverse_active_flat` double,`reverse_active_valley` double,`combined_reactive_total` double,`combined_reactive
_tip` double,`combined_reactive_peak` double,`combined_reactive_flat` double,`combined_reactive_valley` double,`positive_reactive_total` double,`positive_reactive_tip` double,`positive_reactive_peak` double,`positiv
e_reactive_flat` double,`positive_reactive_valley` double,`reverse_reactive_total` double,`reverse_reactive_tip` double,`reverse_reactive_peak` double,`reverse_reactive_flat` double,`reverse_reactive_valley` double)
 TAGS(`meter_code` binary(32),`station_code` binary(8),`meter_type` binary(32));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`strategy_execute`(`ts` timestamp,`execute_time` timestamp,`target_power` double,`active_power` double,`load_power` double,`work_state` t
inyint,`soc` float) TAGS(`station_code` binary(8),`data_month` int);
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`energy_statistics`(`ts` timestamp,`discharged_energy` double,`charged_energy` double,`write_time` timestamp) TAGS(`data_tag` binary(32),
`station_id` int,`cycle_type` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`bmu_data`(`ts` timestamp,`order_no` int,`report_time` timestamp,`write_time` timestamp,`soc` float,`soh` float,`voltage` float,`current`
 float,`bcurrent` float,`online_state` tinyint,`temp_data` binary(500),`volt_data` binary(500),`balance_no` int,`fan_ctrl_pwm` int,`fan_ctrl_state` bigint,`balance_state` bigint,`cell_avg_vol` float,`cell_avg_temp` 
float,`vmax` float,`vmin` float,`vmax_idx` smallint,`vmin_idx` smallint,`tmax` float,`tmin` float,`tmax_idx` smallint,`tmin_idx` smallint,`bmu_balances` binary(512)) TAGS(`bmu_code` binary(32),`cluster_code` binary(
32),`bms_code` binary(16),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`lc_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`online_state` tinyint,`lc_run` tinyint,`lc_fault` tinyint,`main_
breaker_state` tinyint,`main_breaker_fault` tinyint,`bms_state` tinyint,`sys_run_state` bigint,`active_power` float,`reactive_power` float,`power_dc` float,`able_max_chg_power` float,`able_max_dchg_power` float,`rat
ed_power` float,`rated_capacity` float,`power_mode` tinyint,`passive_power` float) TAGS(`lc_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`bms_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`vmax` double,`vmax_bc_idx` smallint,`vmax_bmu_idx` smallint,`vm
ax_cell_idx` smallint,`vmin` double,`vmin_bc_idx` smallint,`vmin_bmu_idx` smallint,`vmin_cell_idx` smallint,`tmax` float,`tmax_bc_idx` smallint,`tmax_bmu_idx` smallint,`tmax_cell_idx` smallint,`tmin` float,`tmin_bc_
idx` smallint,`tmin_bmu_idx` smallint,`tmin_cell_idx` smallint,`current` double,`voltage` double,`power` double,`soc` float,`soh` float,`volt_avg` double,`temp_avg` double,`cur_avg` double,`warn_state` bigint,`prt_s
tate` bigint,`act_state` bigint,`pwr_limit_state` bigint,`sys_state` bigint,`balance_enable_state` bigint,`bc_online_flg` bigint,`left_energy` double,`used_energy` double,`input_energy` double,`output_energy` double
,`input_ah` double,`output_ah` double,`calc_ba_soc` bigint,`show_ba_cal_flag` bigint,`show_ba_soc` bigint,`total_brk_state` bigint,`bc_brk_map_state` bigint,`bc_brk_fault_state` bigint,`chg_pwr_limit` double,`dis_ch
g_pwr_limit` double,`correct` tinyint) TAGS(`bms_code` binary(16),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`device_battery_overview`(`ts` timestamp,`discharged_energy` double,`charged_energy` double,`soc` float) TAGS(`box_code` binary(16),`stat
ion_code` binary(8),`box_type` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`event_msg`(`ts` timestamp,`event_time` timestamp,`collect_time` timestamp,`report_time` timestamp,`event_name` nchar(128),`en_name` bina
ry(256),`dev_num` smallint,`code` int,`senior_type` int,`junior_type` int,`level` tinyint,`event_type` tinyint,`happen_time` timestamp,`disappear_time` timestamp,`event_desc` nchar(512),`en_desc` binary(512),`event_
state` tinyint,`location` binary(512),`params` binary(64),`cap_num` smallint,`device_type` binary(16),`box_code` binary(16),`bms_code` binary(16),`rpt_status` int,`act_flg` int,`confirm` tinyint,`event_bits` tinyint
) TAGS(`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`fire_detector_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`enable_state` tinyint,`online_state` tinyint,`co` flo
at,`voc` float,`temp` float,`smoke` float) TAGS(`fire_detector_code` binary(32),`box_code` binary(16),`station_code` binary(8));
DEBG: dumpInOneDebugFile() LN10291, cmd: CREATE TABLE IF NOT EXISTS ess_cloud.`glc_data`(`ts` timestamp,`report_time` timestamp,`write_time` timestamp,`cfg_lc_sum` smallint) TAGS(`glc_code` binary(32),`station_code`
 binary(8));
INFO: dumpInAvroWorkThreads() dump in avro-tbtags files ...
DEBG: 12 .avro-tbtags files found!
DEBG: 12 .avro-tbtags files filled to list!
INFO: [0]: Restoring from ess_cloud.3438732572185.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732572185.avro-tbtags
INFO: [1]: Restoring from ess_cloud.3438732572316.avro-tbtags ...
INFO: [2]: Restoring from ess_cloud.3438732572343.avro-tbtags ...
INFO: [4]: Restoring from ess_cloud.3438732604114.avro-tbtags ...
INFO: [3]: Restoring from ess_cloud.3438732602635.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732572343.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732572316.avro-tbtags
INFO: [5]: Restoring from ess_cloud.3438732604138.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732602635.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732604114.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732604138.avro-tbtags
INFO: [6]: Restoring from ess_cloud.3438732604175.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732604175.avro-tbtags
INFO: [7]: Restoring from ess_cloud.3438732607035.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732607035.avro-tbtags
INFO: [8]: Restoring from ess_cloud.3438732607108.avro-tbtags ...
INFO: [9]: Restoring from ess_cloud.3438732609974.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732607108.avro-tbtags
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732609974.avro-tbtags
INFO: [10]: Restoring from ess_cloud.3438732612378.avro-tbtags ...
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732612378.avro-tbtags
INFO: [11]: Restoring from ess_cloud.3438732612455.avro-tbtags ...
DEBG: avroFile: td-bak/taosdump.3438732572160/ess_cloud.3438732612455.avro-tbtags
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud
DEBG: dumpInOneAvroFile() LN7522, Namespace: ess_cloud

运行环境:

  • CentOS 7.9
  • Memory: 7.6G, CPU: Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz * 4, current Disk Space: 184G
  • TDengine Version 3.2.3.0
@Whitroom Whitroom added the bug Something isn't working label Jun 26, 2024
@Whitroom Whitroom changed the title taosdump 3.2.3.0 import stuck Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
1 participant
@Whitroom