HADOOP-19228. ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'. #6936

fuchaohong · 2024-07-10T08:56:20Z

JIRA: HADOOP-19228.
When setting configuration into environment variables, '-' should also be replaced with '_'.

hadoop-yetus · 2024-07-10T12:33:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 13s		trunk passed
+1 💚	compile	16m 58s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	15m 44s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	1m 15s		trunk passed
+1 💚	mvnsite	1m 41s		trunk passed
+1 💚	javadoc	1m 11s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	0m 51s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	2m 28s		trunk passed
+1 💚	shadedclient	34m 46s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 55s		the patch passed
+1 💚	compile	16m 16s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	16m 16s		the patch passed
+1 💚	compile	15m 25s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	15m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 9s		the patch passed
+1 💚	mvnsite	1m 36s		the patch passed
+1 💚	javadoc	1m 5s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	0m 49s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	2m 37s		the patch passed
+1 💚	shadedclient	35m 48s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 22s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 59s		The patch does not generate ASF License warnings.
		216m 5s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6936/1/artifact/out/Dockerfile
GITHUB PR	#6936
JIRA Issue	HDFS-17578
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux c35ee37e1929 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `be91412`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6936/1/testReport/
Max. process+thread count	1309 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6936/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao · 2024-07-11T14:05:58Z

Sorry I didn't get what this PR want to do, do you meet any issues here? Thanks.

fuchaohong · 2024-07-12T01:35:58Z

Using the shell fencing method.
dfs.ha.fencing.methods=shell(/path/fence.sh)
dfs.ha.fencing.ssh.connect-timeout=30000
The environment variable dfs_ha_fencing_ssh_connect-timeout cannot be used in /path/fence.sh and should be named dfs_ha_fencing_ssh_connect_timeout.
@Hexiaoqiao, Thanks.

Hexiaoqiao · 2024-07-16T12:48:27Z

Got it. We should update both logic (include javadocs) of replace special character at ShellCommandFencer. Another one, we should move module from HDFS to COMMON. Thanks.

fuchaohong · 2024-07-17T03:14:50Z

@Hexiaoqiao Thank you for your review. The code has been updated. Please review it again. Thanks.

hadoop-yetus · 2024-07-17T06:36:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	45m 45s		trunk passed
+1 💚	compile	17m 34s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	16m 19s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	1m 19s		trunk passed
+1 💚	mvnsite	1m 46s		trunk passed
+1 💚	javadoc	1m 17s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	0m 54s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	2m 37s		trunk passed
+1 💚	shadedclient	35m 47s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 56s		the patch passed
+1 💚	compile	16m 52s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	16m 52s		the patch passed
+1 💚	compile	16m 10s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	16m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 13s		the patch passed
+1 💚	mvnsite	1m 43s		the patch passed
+1 💚	javadoc	1m 11s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	0m 55s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	2m 47s		the patch passed
+1 💚	shadedclient	36m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 40s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 5s		The patch does not generate ASF License warnings.
		223m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6936/2/artifact/out/Dockerfile
GITHUB PR	#6936
JIRA Issue	HDFS-17578
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 48d7d4926fc3 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ca48b8f`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6936/2/testReport/
Max. process+thread count	1263 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6936/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

LGTM. +1.

Hexiaoqiao · 2024-07-20T08:13:49Z

Committed to trunk. Thanks @fuchaohong for your works.

ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'.

be91412

github-actions bot added trunk Common labels Jul 10, 2024

update ShellCommandFencer

ca48b8f

fuchaohong changed the title ~~HDFS-17578: ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'.~~ Jul 17, 2024

Hexiaoqiao changed the title ~~HADOOP-19228: ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'.~~ Jul 20, 2024

Hexiaoqiao approved these changes Jul 20, 2024

View reviewed changes

Hexiaoqiao merged commit 1577f57 into apache:trunk Jul 20, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-19228. ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'. #6936

HADOOP-19228. ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'. #6936

fuchaohong commented Jul 10, 2024 •

edited

Loading

hadoop-yetus commented Jul 10, 2024

Hexiaoqiao commented Jul 11, 2024

fuchaohong commented Jul 12, 2024 •

edited

Loading

Hexiaoqiao commented Jul 16, 2024

fuchaohong commented Jul 17, 2024

hadoop-yetus commented Jul 17, 2024

Hexiaoqiao left a comment

Hexiaoqiao commented Jul 20, 2024

HADOOP-19228. ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'. #6936

HADOOP-19228. ShellCommandFencer#setConfAsEnvVars should also replace '-' with '_'. #6936

Conversation

fuchaohong commented Jul 10, 2024 • edited Loading

hadoop-yetus commented Jul 10, 2024

Hexiaoqiao commented Jul 11, 2024

fuchaohong commented Jul 12, 2024 • edited Loading

Hexiaoqiao commented Jul 16, 2024

fuchaohong commented Jul 17, 2024

hadoop-yetus commented Jul 17, 2024

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Hexiaoqiao commented Jul 20, 2024

fuchaohong commented Jul 10, 2024 •

edited

Loading

fuchaohong commented Jul 12, 2024 •

edited

Loading