Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](auto bucket) Fix hit not support alter estimate_partition_size #33670 #37587

Merged

Conversation

deardeng
Copy link
Contributor

cherry pick from #33670

@deardeng
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Jul 10, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 50001 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d76628d59106c1a86d2805e579583df0a6e980b9, data reload: false

------ Round 1 ----------------------------------
q1	17638	4488	4413	4413
q2	2097	152	147	147
q3	10475	1910	1915	1910
q4	10325	1265	1356	1265
q5	8562	3923	3897	3897
q6	237	150	128	128
q7	2055	1583	1605	1583
q8	9310	2760	2731	2731
q9	10651	10514	10284	10284
q10	8666	3515	3543	3515
q11	431	257	246	246
q12	475	299	318	299
q13	18343	3964	4057	3964
q14	363	335	344	335
q15	512	451	452	451
q16	672	577	575	575
q17	1125	984	976	976
q18	7214	6765	6895	6765
q19	1761	1690	1614	1614
q20	530	319	310	310
q21	4492	4173	4152	4152
q22	540	441	442	441
Total cold run time: 116474 ms
Total hot run time: 50001 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4351	4324	4348	4324
q2	322	225	228	225
q3	4174	4176	4167	4167
q4	2758	2741	2740	2740
q5	7147	7093	7119	7093
q6	244	119	118	118
q7	3255	2855	2856	2855
q8	4376	4472	4501	4472
q9	16843	16795	16777	16777
q10	4268	4280	4304	4280
q11	755	693	675	675
q12	1032	868	886	868
q13	6985	3751	3756	3751
q14	452	414	429	414
q15	512	455	453	453
q16	749	671	676	671
q17	3784	3881	3865	3865
q18	8909	8893	8766	8766
q19	1717	1734	1698	1698
q20	2409	2110	2128	2110
q21	8506	8504	8539	8504
q22	1058	930	981	930
Total cold run time: 84606 ms
Total hot run time: 79756 ms
@doris-robot
Copy link

TPC-DS: Total hot run time: 203441 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d76628d59106c1a86d2805e579583df0a6e980b9, data reload: false

query1	932	423	380	380
query2	6524	2850	2495	2495
query3	6955	207	198	198
query4	20650	18006	18083	18006
query5	19725	6538	6525	6525
query6	295	226	241	226
query7	4163	310	303	303
query8	440	432	449	432
query9	3070	2652	2585	2585
query10	425	314	309	309
query11	11255	10692	10682	10682
query12	129	73	78	73
query13	5618	718	702	702
query14	17753	14033	13601	13601
query15	355	244	245	244
query16	6475	293	264	264
query17	1723	1454	880	880
query18	2314	420	412	412
query19	219	152	155	152
query20	85	84	85	84
query21	197	97	94	94
query22	5194	4948	5008	4948
query23	32383	31984	32093	31984
query24	7017	6604	6504	6504
query25	519	445	442	442
query26	537	172	164	164
query27	1860	299	301	299
query28	6092	2360	2329	2329
query29	2951	2652	2750	2652
query30	243	167	178	167
query31	924	749	744	744
query32	69	64	65	64
query33	398	260	254	254
query34	826	485	473	473
query35	1117	916	895	895
query36	1297	1147	1285	1147
query37	96	61	62	61
query38	3114	2929	2929	2929
query39	1368	1345	1327	1327
query40	222	88	97	88
query41	46	43	45	43
query42	92	92	85	85
query43	696	645	750	645
query44	1162	741	732	732
query45	250	241	237	237
query46	1226	957	992	957
query47	1774	1673	1768	1673
query48	1010	758	710	710
query49	615	375	369	369
query50	860	652	633	633
query51	4782	4626	4714	4626
query52	93	85	74	74
query53	451	322	321	321
query54	2630	2454	2457	2454
query55	86	83	85	83
query56	237	215	206	206
query57	1156	1145	1113	1113
query58	231	212	198	198
query59	4443	4199	4126	4126
query60	210	200	211	200
query61	100	99	102	99
query62	827	445	454	445
query63	496	343	347	343
query64	2530	1577	1440	1440
query65	3606	3571	3585	3571
query66	790	389	384	384
query67	16202	15674	14662	14662
query68	10352	664	631	631
query69	570	355	355	355
query70	1863	1408	1304	1304
query71	430	316	336	316
query72	6670	3489	3503	3489
query73	730	319	330	319
query74	6374	5808	5835	5808
query75	5473	3730	3724	3724
query76	6648	1146	1179	1146
query77	1146	252	257	252
query78	12525	12565	11856	11856
query79	11792	641	622	622
query80	746	407	399	399
query81	495	234	234	234
query82	1056	100	99	99
query83	180	135	133	133
query84	261	73	70	70
query85	828	350	341	341
query86	330	288	303	288
query87	3240	3046	3018	3018
query88	5516	2338	2345	2338
query89	425	307	295	295
query90	2622	210	215	210
query91	193	153	138	138
query92	56	54	52	52
query93	6431	612	538	538
query94	1362	213	210	210
query95	1141	1087	1065	1065
query96	642	333	322	322
query97	6397	6317	6544	6317
query98	191	175	179	175
query99	2895	945	830	830
Total cold run time: 321710 ms
Total hot run time: 203441 ms
@doris-robot
Copy link

ClickBench: Total hot run time: 30.65 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d76628d59106c1a86d2805e579583df0a6e980b9, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.03
query3	0.24	0.04	0.05
query4	1.79	0.05	0.06
query5	0.56	0.52	0.51
query6	1.23	0.67	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.52	0.47	0.48
query10	0.55	0.54	0.54
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.63	0.61	0.62
query14	0.77	0.78	0.82
query15	0.78	0.77	0.75
query16	0.39	0.37	0.36
query17	1.03	0.99	1.03
query18	0.24	0.24	0.26
query19	1.91	1.87	1.87
query20	0.02	0.00	0.01
query21	15.44	0.55	0.57
query22	2.06	2.05	1.57
query23	17.24	1.05	0.98
query24	7.16	0.99	0.84
query25	0.37	0.08	0.06
query26	0.74	0.15	0.16
query27	0.03	0.05	0.04
query28	6.25	0.79	0.78
query29	12.77	2.34	2.32
query30	0.60	0.52	0.55
query31	2.81	0.39	0.38
query32	3.37	0.49	0.49
query33	3.03	3.14	3.05
query34	15.28	4.80	4.75
query35	4.86	4.83	4.84
query36	1.04	1.01	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.08	0.01	0.01
query42	0.03	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.5 s
Total hot run time: 30.65 s
@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d76628d59106c1a86d2805e579583df0a6e980b9 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s
@dataroaring dataroaring merged commit 68c4ce1 into apache:branch-2.0 Jul 10, 2024
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/planner Issues or PRs related to the query planner
3 participants