feat: AI Proxy, AWS Bedrock Converse-API Driver #13354
Conversation
github-actions
bot
added
schema-change-noteworthy
cherry-pick kong-ee
|
@fffonion let's start the review process on this large PR. |
tysoekong
force-pushed
the
feat/ai_proxy_aws_bedrock
branch
from
2c5883a
to
1a541a3
Compare
tysoekong
force-pushed
the
feat/ai_proxy_aws_bedrock
branch
from
dc5b8ae
to
234b3af
Compare
tysoekong
force-pushed
the
feat/ai_proxy_aws_bedrock
branch
from
234b3af
to
fa52c16
Compare
tysoekong
force-pushed
the
feat/ai_proxy_aws_bedrock
branch
from
fa52c16
to
f499d3f
Compare
kong/clustering/compat/checkers.lua
Outdated
| has_update = true | ||
| end | ||
| end | ||
|
|
||
| if plugin.name == 'ai-request-transformer' then | ||
| local config = plugin.config | ||
| if config.llm.model.provider == "gemini" then | ||
| config.llm.model.provider = "openai" | ||
| if config.llm.model.provider == "gemini" or config.llm.model.provider == "bedrock" then |
There was a problem hiding this comment.
You also seem to do quite a bit of comparison for model/providers. Consider doing something like
local GEMINI_PROVIDER_NAME = "gemini"
is_gemini = function()
return config.llm.model.provider == GEMINI_PROVIDER_NAME
end
-- allows you to do compounds as well
is_gemini_or_bedrock() = funtion()
return is_gemini() or is_bedrock()
end
There was a problem hiding this comment.
But will this JIT really badly, and have worse performance? Is there anywhere else I can copy from for this?
There was a problem hiding this comment.
What about this?
Keeps strings together, and also jits all strings at startup (when clustering package launches).
| @@ -219,7 +243,7 @@ end | |||
| -- @param {string} frame input string to format into SSE events | |||
| -- @param {boolean} raw_json sets application/json byte-parser mode | |||
| -- @return {table} n number of split SSE messages, or empty table | |||
| function _M.frame_to_events(frame, raw_json_mode) | |||
There was a problem hiding this comment.
Are you sure that you patched all frame_to_events invocations. This is a shared library and there may be some false expectations when the function signature changes.
There was a problem hiding this comment.
Yeah checked all providers. This won't happen again after this release, will be moved into their respective drivers.
I have no idea why I put this function in shared in the first place...
tysoekong
force-pushed
the
feat/ai_proxy_aws_bedrock
branch
from
9d864e7
to
f704158
Compare
Summary
Supersedes #13054 which was completely broken.
Adds AWS Bedrock "Converse API" support to Kong AI Gateway.
Checklist
changelog/unreleased/kongorskip-changeloglabel added on PR if changelog is unnecessary. README.mdIssue reference
AG-14