feat(*): port several PDK and shared llm module from ai-proxy-advanced feature #13347
Conversation
github-actions
bot
added
core/balancer
core/pdk
cherry-pick kong-ee
fffonion
added
skip-changelog
and removed
cherry-pick kong-ee
AndyZhang0707
requested review from
samugi,
liverpool8056 and
jschmid1
and removed request for
samugi
| @@ -359,6 +359,16 @@ local function execute(balancer_data, ctx) | |||
| balancer_data.balancer_handle = handle | |||
|
|
|||
| else | |||
| -- Note: balancer_data.retry_callback is only set by PDK once in access phase | |||
There was a problem hiding this comment.
It seems that the retry_callback can be invoked only when balancer == nil, is it expected to work like this?
In addition, should we wrap retry_callback with pcall?
There was a problem hiding this comment.
It seems that the retry_callback can be invoked only when balancer == nil, is it expected to work like this?
Yes, if we skip the balancer lookup by doing kong.service.set_target, i.e. currently request doesn't have a upstream, then balancer, upstream = balancers.get_balancer(balancer_data) will return no balancer. Actually, this else branch is just made to support kong.service.set_target.
This matches our expectation, retry_callback will not be called if Kong's balancer is not skipped.
In addition, should we wrap retry_callback with pcall?
yes, adding that now.
There was a problem hiding this comment.
I will address all comments in EE side of PR and port here
|
Reviewers: please review https://github.com/Kong/kong-ee/pull/9562 first for a whole picture, I will then address comments there and port here. |
github-actions
bot
added
the
cherry-pick kong-ee
Summary
This PR adds several PDK methods to allow ai-proxy-advanced plugin to do customized load balancing and failover.
It also moves the ai-proxy handler to a shared module so that both ai-proxy{,-advanced} plugins can reuse same codebase.
https://github.com/Kong/kong-ee/pull/9562
Checklist
changelog/unreleased/kongorskip-changeloglabel added on PR if changelog is unnecessary. README.mdIssue reference
AG-12