Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](partition) Fix be tablet partition id eq 0 By report tablet #32179 #37588

Merged
merged 2 commits into from
Jul 13, 2024
Merged

[fix](partition) Fix be tablet partition id eq 0 By report tablet #32179 #37588

merged 2 commits into from
Jul 13, 2024

Conversation

deardeng
Copy link
Contributor

cherry pick from #32179
@deardeng
Copy link
Contributor Author

run buildall
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.89% (8117/21420)
Line Coverage: 29.56% (66512/225028)
Region Coverage: 29.03% (34279/118094)
Branch Coverage: 24.90% (17608/70722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/642520095489924f4c9ce1196447886d8e25170d_642520095489924f4c9ce1196447886d8e25170d/report/index.html
@doris-robot
Copy link

TPC-H: Total hot run time: 49867 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 642520095489924f4c9ce1196447886d8e25170d, data reload: false

------ Round 1 ----------------------------------
q1	17652	4423	4363	4363
q2	2076	157	146	146
q3	10455	1928	1936	1928
q4	10324	1238	1307	1238
q5	8766	3851	3888	3851
q6	233	128	123	123
q7	2029	1588	1607	1588
q8	9311	2736	2715	2715
q9	10513	10392	10222	10222
q10	8637	3565	3492	3492
q11	415	239	253	239
q12	476	301	310	301
q13	18343	4003	3986	3986
q14	365	327	333	327
q15	511	450	466	450
q16	682	575	576	575
q17	1132	974	960	960
q18	7202	7044	6885	6885
q19	1800	1631	1632	1631
q20	537	305	306	305
q21	4451	4115	4126	4115
q22	529	439	427	427
Total cold run time: 116439 ms
Total hot run time: 49867 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4323	4362	4307	4307
q2	326	222	223	222
q3	4143	4128	4123	4123
q4	2764	2766	2736	2736
q5	7131	7094	7066	7066
q6	239	122	122	122
q7	3255	2840	2806	2806
q8	4349	4512	4481	4481
q9	16916	16875	16668	16668
q10	4230	4220	4272	4220
q11	770	712	677	677
q12	1011	832	866	832
q13	7107	3766	3761	3761
q14	457	418	425	418
q15	504	448	455	448
q16	733	684	693	684
q17	3846	3853	3875	3853
q18	8943	8807	8777	8777
q19	1751	1707	1688	1688
q20	2363	2095	2121	2095
q21	8571	8417	8558	8417
q22	1033	956	987	956
Total cold run time: 84765 ms
Total hot run time: 79357 ms
@doris-robot
Copy link

TPC-DS: Total hot run time: 203264 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 642520095489924f4c9ce1196447886d8e25170d, data reload: false

query1	936	420	376	376
query2	6524	2844	2529	2529
query3	6917	213	201	201
query4	20337	18097	18044	18044
query5	19747	6525	6429	6429
query6	298	221	224	221
query7	4154	311	318	311
query8	448	437	410	410
query9	3076	2661	2583	2583
query10	415	312	297	297
query11	11375	10658	10621	10621
query12	129	81	78	78
query13	5601	701	693	693
query14	17311	13528	13431	13431
query15	406	249	245	245
query16	6482	287	262	262
query17	1754	1450	898	898
query18	2296	408	414	408
query19	212	153	142	142
query20	74	78	80	78
query21	195	95	92	92
query22	5372	5025	5110	5025
query23	32768	31915	31874	31874
query24	7070	6493	6506	6493
query25	537	426	420	420
query26	528	164	167	164
query27	1797	309	297	297
query28	6116	2332	2290	2290
query29	2885	2817	2759	2759
query30	242	166	165	165
query31	910	761	756	756
query32	71	67	58	58
query33	383	254	245	245
query34	859	473	477	473
query35	1126	913	918	913
query36	1315	1144	1264	1144
query37	90	59	62	59
query38	3057	2924	2950	2924
query39	1394	1332	1311	1311
query40	203	95	99	95
query41	46	44	44	44
query42	87	87	90	87
query43	687	893	745	745
query44	1135	722	735	722
query45	244	235	235	235
query46	1231	964	969	964
query47	1696	1694	1887	1694
query48	1017	723	709	709
query49	619	368	376	368
query50	873	565	558	558
query51	4756	4609	4662	4609
query52	100	91	81	81
query53	452	336	336	336
query54	2669	2469	2477	2469
query55	90	88	83	83
query56	241	228	222	222
query57	1151	1058	1103	1058
query58	236	209	195	195
query59	4356	3832	4162	3832
query60	212	219	226	219
query61	98	97	94	94
query62	828	442	449	442
query63	487	347	347	347
query64	2310	1534	1503	1503
query65	3630	3564	3570	3564
query66	803	394	378	378
query67	15638	15581	15012	15012
query68	10346	656	637	637
query69	558	338	348	338
query70	1824	1464	1348	1348
query71	418	308	316	308
query72	6647	3537	3502	3502
query73	741	325	323	323
query74	6314	5780	5934	5780
query75	5294	3772	3709	3709
query76	6698	1111	1220	1111
query77	1178	254	250	250
query78	12533	11786	12769	11786
query79	10690	627	634	627
query80	1030	394	394	394
query81	494	236	234	234
query82	660	93	99	93
query83	168	136	136	136
query84	253	70	72	70
query85	883	325	331	325
query86	346	295	292	292
query87	3308	3032	3036	3032
query88	4399	2325	2343	2325
query89	370	285	290	285
query90	2014	214	219	214
query91	172	141	156	141
query92	59	53	54	53
query93	4411	553	560	553
query94	756	210	212	210
query95	1097	1043	1054	1043
query96	645	331	331	331
query97	6547	6355	6432	6355
query98	182	176	173	173
query99	3044	914	902	902
Total cold run time: 315181 ms
Total hot run time: 203264 ms
@doris-robot
Copy link

ClickBench: Total hot run time: 30.32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 642520095489924f4c9ce1196447886d8e25170d, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.04	0.05
query4	1.80	0.07	0.06
query5	0.55	0.53	0.52
query6	1.22	0.62	0.60
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.53	0.49	0.48
query10	0.53	0.53	0.55
query11	0.12	0.09	0.08
query12	0.11	0.09	0.09
query13	0.62	0.61	0.60
query14	0.78	0.78	0.79
query15	0.78	0.76	0.76
query16	0.37	0.37	0.37
query17	1.01	0.99	1.01
query18	0.22	0.26	0.26
query19	1.87	1.88	1.82
query20	0.02	0.01	0.01
query21	15.46	0.55	0.55
query22	2.21	1.90	1.70
query23	17.24	0.94	1.06
query24	5.92	7.17	0.47
query25	0.33	0.16	0.04
query26	1.13	0.16	0.15
query27	0.05	0.04	0.04
query28	3.16	0.76	0.72
query29	12.68	2.30	2.28
query30	0.57	0.52	0.53
query31	2.81	0.38	0.38
query32	3.38	0.50	0.49
query33	3.07	3.08	3.07
query34	15.28	4.81	4.80
query35	4.85	4.84	4.87
query36	1.06	1.02	1.02
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 100.52 s
Total hot run time: 30.32 s
@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 642520095489924f4c9ce1196447886d8e25170d with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s
@deardeng deardeng force-pushed the pick_32179_to_doris_branch-2.0 branch from 6425200 to ee53571 Compare July 12, 2024 08:02
@deardeng
Copy link
Contributor Author

run buildall
@doris-robot
Copy link

TPC-H: Total hot run time: 51356 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ee53571c74df66462676751bafc1103537d70398, data reload: false

------ Round 1 ----------------------------------
q1	17599	4620	4548	4548
q2	2068	161	154	154
q3	10459	2007	2047	2007
q4	10284	1323	1447	1323
q5	8501	3913	4019	3913
q6	280	130	125	125
q7	2104	1582	1596	1582
q8	9292	2909	2905	2905
q9	10805	10509	10430	10430
q10	8663	3592	3546	3546
q11	425	249	244	244
q12	514	311	308	308
q13	18381	4070	4108	4070
q14	352	332	327	327
q15	520	461	459	459
q16	688	571	577	571
q17	1193	1013	1042	1013
q18	7298	7029	7086	7029
q19	1874	1758	1735	1735
q20	542	301	317	301
q21	4703	4326	4340	4326
q22	550	440	450	440
Total cold run time: 117095 ms
Total hot run time: 51356 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4552	4494	4512	4494
q2	338	232	224	224
q3	4225	4215	4248	4215
q4	2857	2852	2860	2852
q5	7033	6997	6995	6995
q6	259	120	122	120
q7	3255	2795	2819	2795
q8	4242	4413	4399	4399
q9	16988	16922	16918	16918
q10	4306	4338	4300	4300
q11	726	663	681	663
q12	1054	835	869	835
q13	6970	3883	3869	3869
q14	459	435	437	435
q15	516	455	455	455
q16	738	678	686	678
q17	3790	3860	3814	3814
q18	8872	8871	8845	8845
q19	1821	1850	1806	1806
q20	2386	2117	2111	2111
q21	8564	8440	8470	8440
q22	1081	993	934	934
Total cold run time: 85032 ms
Total hot run time: 80197 ms
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.89% (8116/21422)
Line Coverage: 29.54% (66530/225194)
Region Coverage: 29.01% (34285/118186)
Branch Coverage: 24.88% (17610/70786)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ee53571c74df66462676751bafc1103537d70398_ee53571c74df66462676751bafc1103537d70398/report/index.html
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"
@doris-robot
Copy link

TPC-DS: Total hot run time: 203480 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ee53571c74df66462676751bafc1103537d70398, data reload: false

query1	916	415	376	376
query2	6522	2813	2669	2669
query3	6921	202	205	202
query4	20168	18138	17871	17871
query5	19739	6520	6512	6512
query6	305	217	230	217
query7	4158	298	308	298
query8	434	453	415	415
query9	3078	2691	2611	2611
query10	419	298	294	294
query11	11285	10812	10636	10636
query12	128	77	74	74
query13	5604	694	707	694
query14	17745	13132	13179	13132
query15	358	244	248	244
query16	6472	295	262	262
query17	1686	1459	880	880
query18	2310	423	399	399
query19	204	149	148	148
query20	83	75	80	75
query21	194	97	95	95
query22	5270	4814	4955	4814
query23	32505	31828	31918	31828
query24	6973	6451	6485	6451
query25	519	420	403	403
query26	531	164	167	164
query27	1892	289	292	289
query28	6163	2402	2358	2358
query29	2932	2784	2845	2784
query30	252	164	167	164
query31	914	725	731	725
query32	65	64	63	63
query33	398	257	248	248
query34	855	469	490	469
query35	1129	929	933	929
query36	1437	1124	1228	1124
query37	94	63	60	60
query38	3073	2947	2905	2905
query39	1378	1326	1319	1319
query40	202	99	95	95
query41	47	43	43	43
query42	82	91	90	90
query43	870	628	749	628
query44	1132	736	741	736
query45	247	233	232	232
query46	1219	956	968	956
query47	2113	1668	1643	1643
query48	1019	719	698	698
query49	625	365	360	360
query50	869	586	580	580
query51	4805	4706	4690	4690
query52	94	87	96	87
query53	437	321	320	320
query54	2673	2468	2430	2430
query55	98	72	87	72
query56	218	233	209	209
query57	1193	1120	1160	1120
query58	205	203	198	198
query59	4143	3644	4412	3644
query60	207	206	188	188
query61	96	91	95	91
query62	838	536	479	479
query63	491	339	341	339
query64	2485	1541	1378	1378
query65	3604	3584	3523	3523
query66	775	390	375	375
query67	15580	15222	15013	15013
query68	11156	669	677	669
query69	614	332	360	332
query70	2058	1359	1343	1343
query71	439	313	313	313
query72	6478	3495	3493	3493
query73	751	326	324	324
query74	6352	5840	5817	5817
query75	5616	3702	3650	3650
query76	6979	1112	1168	1112
query77	1189	264	260	260
query78	12746	13118	13096	13096
query79	8895	644	649	644
query80	667	407	398	398
query81	453	234	227	227
query82	243	97	101	97
query83	186	135	135	135
query84	258	70	68	68
query85	734	325	329	325
query86	349	291	292	291
query87	3268	3035	3037	3035
query88	4589	2330	2357	2330
query89	345	309	287	287
query90	2323	206	214	206
query91	183	157	138	138
query92	62	52	52	52
query93	1939	595	588	588
query94	926	204	212	204
query95	1122	1047	1064	1047
query96	635	325	330	325
query97	6534	6237	6316	6237
query98	188	175	173	173
query99	2986	920	846	846
Total cold run time: 312739 ms
Total hot run time: 203480 ms
@doris-robot
Copy link

ClickBench: Total hot run time: 30.43 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ee53571c74df66462676751bafc1103537d70398, data reload: false

query1	0.03	0.02	0.02
query2	0.08	0.02	0.02
query3	0.25	0.04	0.05
query4	1.79	0.06	0.05
query5	0.54	0.53	0.53
query6	1.25	0.66	0.61
query7	0.01	0.00	0.01
query8	0.04	0.03	0.02
query9	0.53	0.48	0.47
query10	0.54	0.53	0.53
query11	0.12	0.08	0.08
query12	0.11	0.09	0.09
query13	0.62	0.62	0.60
query14	0.77	0.78	0.78
query15	0.78	0.76	0.75
query16	0.38	0.36	0.37
query17	1.01	1.03	1.04
query18	0.22	0.27	0.22
query19	1.92	1.82	1.84
query20	0.01	0.01	0.02
query21	15.46	0.57	0.53
query22	2.01	1.96	1.47
query23	17.07	0.94	1.01
query24	4.47	0.82	0.98
query25	0.27	0.06	0.06
query26	0.63	0.15	0.15
query27	0.05	0.04	0.03
query28	9.56	0.71	0.72
query29	12.64	2.29	2.35
query30	0.62	0.53	0.52
query31	2.85	0.39	0.38
query32	3.39	0.50	0.49
query33	3.05	3.11	3.12
query34	15.26	4.81	4.80
query35	4.89	4.84	4.85
query36	1.06	1.01	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.13
query41	0.07	0.01	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.67 s
Total hot run time: 30.43 s
@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ee53571c74df66462676751bafc1103537d70398 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s
@dataroaring dataroaring merged commit 15a94ba into apache:branch-2.0 Jul 13, 2024
21 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants
@deardeng @doris-robot @dataroaring