-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DSIP-56] Refactor JDBC registry support session timeout and data change event #16287
[DSIP-56] Refactor JDBC registry support session timeout and data change event #16287
Conversation
...g/apache/dolphinscheduler/plugin/registry/jdbc/model/DTO/JdbcRegistryClientHeartbeatDTO.java
Fixed
Show fixed
Hide fixed
...g/apache/dolphinscheduler/plugin/registry/jdbc/model/DTO/JdbcRegistryClientHeartbeatDTO.java
Fixed
Show fixed
Hide fixed
...in/java/org/apache/dolphinscheduler/plugin/registry/jdbc/server/JdbcRegistryDataManager.java
Fixed
Show fixed
Hide fixed
...rc/main/java/org/apache/dolphinscheduler/plugin/registry/jdbc/server/JdbcRegistryServer.java
Fixed
Show fixed
Hide fixed
.build()); | ||
return; | ||
} | ||
log.debug("{} acquire the lock {} success", lockOwner, lockKey); |
Check failure
Code scanning / CodeQL
Log Injection High
user-provided value
.build()); | ||
return true; | ||
} | ||
log.debug("{} acquire the lock {} success", lockOwner, lockKey); |
Check failure
Code scanning / CodeQL
Log Injection High
user-provided value
...g/apache/dolphinscheduler/plugin/registry/jdbc/model/DTO/JdbcRegistryClientHeartbeatDTO.java
Fixed
Show fixed
Hide fixed
14c7651
to
d2ac15a
Compare
379dae8
to
af80ef4
Compare
11409b9
to
c93008c
Compare
c93008c
to
4106238
Compare
4106238
to
caefdd0
Compare
dolphinscheduler-tools/src/main/java/org/apache/dolphinscheduler/tools/command/ICommand.java
Fixed
Show fixed
Hide fixed
"\n sessionTimeout -> " + sessionTimeout + | ||
"\n hikariConfig -> " + hikariConfig + | ||
"\n****************************JdbcRegistryProperties**************************************"; | ||
log.info(config); |
Check failure
Code scanning / CodeQL
Log Injection High
user-provided value
d53f811
to
1b3c3aa
Compare
...eduler-tools/src/main/java/org/apache/dolphinscheduler/tools/command/CommandApplication.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
.github/workflows/cluster-test/postgresql_with_zookeeper_registry/dolphinscheduler_env.sh
Show resolved
Hide resolved
DEFAULT CHARSET = utf8; | ||
|
||
DROP TABLE IF EXISTS `t_ds_jdbc_registry_client_heartbeat`; | ||
CREATE TABLE `t_ds_jdbc_registry_client_heartbeat` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Heartbeat is just an attribute of client, so maybe registry_client is better.
CREATE TABLE `t_ds_jdbc_registry_client_heartbeat` | |
CREATE TABLE `t_ds_jdbc_registry_client` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The origin table name is t_ds_jdbc_registry_client, but I am worried this will conflict with JdbcRegistryClient which is already exist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's OK because it has t_ds prefix, and what about using DsJdbcRegistryClient
as entity class name?
( | ||
`id` bigint(11) NOT NULL AUTO_INCREMENT COMMENT 'primary key', | ||
`event_type` varchar(64) NOT NULL COMMENT 'ADD, UPDATE, DELETE', | ||
`jdbc_registry_data` text NOT NULL COMMENT 'jdbc registry data', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`jdbc_registry_data` text NOT NULL COMMENT 'jdbc registry data', | |
`registry_data` text NOT NULL COMMENT 'registry data', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use jdbc_registry_data may better to know the data is from t_ds_jdbc_registry_data
@Select("select max(id) from t_ds_jdbc_registry_data_change_event") | ||
Long getMaxId(); | ||
|
||
@Select("select * from t_ds_jdbc_registry_data_change_event where id > #{id} order by id asc limit 1000") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Select("select * from t_ds_jdbc_registry_data_change_event where id > #{id} order by id asc limit 1000") | |
@Select("select * from t_ds_jdbc_registry_data_change_event where id > #{id} limit 1000") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Directly announce the order is better to let others know we need to make sure the result have order here.
|
||
void subscribeRegistryRowChange(RegistryRowChangeListener<T> registryRowChangeListener); | ||
|
||
interface RegistryRowChangeListener<T> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RegistryRow
means RegistryData
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
row is a database concept, and it is best to use logical concepts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not only represent to RegistryData, but now only have RegistryData implemenration, this interface is used to notify the jdbc row add/update/delete, so we can directly use jdbc concepts here.
...in/java/org/apache/dolphinscheduler/plugin/registry/jdbc/server/JdbcRegistryServerState.java
Outdated
Show resolved
Hide resolved
...istry-jdbc/src/test/java/org/apache/dolphinscheduler/plugin/registry/jdbc/LockUtilsTest.java
Outdated
Show resolved
Hide resolved
...st/java/org/apache/dolphinscheduler/plugin/registry/jdbc/PostgresqlJdbcRegistryTestCase.java
Outdated
Show resolved
Hide resolved
ecc3e97
to
c6ead62
Compare
c6ead62
to
f0578f7
Compare
Should update the doc too? see https://dolphinscheduler.apache.org/zh-cn/docs/3.2.1/architecture/configuration |
The jdbc relation configuration doc is link to jdbc registry README.md which will be updated in this PR. |
f0578f7
to
99b6ce1
Compare
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM overall
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Purpose of the pull request
close #16278
Brief change log
Verify this pull request
Test by UT and E2E
Pull Request Notice
Pull Request Notice
If your pull request contain incompatible change, you should also add it to
docs/docs/en/guide/upgrede/incompatible.md