config for local models, switch to NewOpenAIFunctionsAgent#15
config for local models, switch to NewOpenAIFunctionsAgent#15tnthornton merged 2 commits intomainfrom
Conversation
jaymiracola
force-pushed
the
local-model-config
branch
from
0e28f41 to
1242060
Compare
tnthornton
left a comment
tnthornton
left a comment
There was a problem hiding this comment.
Thanks @jaymiracola ! I just have the one comment for something for us to keep an eye on as we're using this new functionality over time 👍
| var baseURL string | ||
| if baseURLBytes, ok := c.Data[credBaseURLKey]; ok { | ||
| baseURL = strings.Trim(string(baseURLBytes), "\n") | ||
| } | ||
|
|
||
| // Extract optional model from credentials, default to gpt-4 | ||
| model := defaultModel | ||
| if modelBytes, ok := c.Data[credModelKey]; ok { | ||
| model = strings.Trim(string(modelBytes), "\n") | ||
| } |
There was a problem hiding this comment.
I think this is generally fine for now. One thing that I'm not sure about is if these configurations should stay in the secret, as roughly a configuration singleton, or if we should move them to the input. Both options are dynamic for the lifespan of the function request, but the model and the base url aren't really "sensitive". Also moving it to the input may make it a little more apparent how the function is configured within the Operation pipeline itself.
Just some things to think about as we try out this new functionality 👍
There was a problem hiding this comment.
Agreed. I struggled with where to put them. For the sake of convenience of uniformity I landed on the secret for now.
jaymiracola
force-pushed
the
local-model-config
branch
from
1242060 to
7b8b122
Compare
…gent for better parsing Signed-off-by: Jay Miracola <jaymiracola@gmail.com>
jaymiracola
force-pushed
the
local-model-config
branch
from
7b8b122 to
086db8c
Compare
2 participants
Description of your changes
Adding configuration ability for local models conforming to OpenAI API standards. Switched to agents.NewOpenAIFunctionsAgent() instead of agents.NewOneShotAgent() to help handle parsing output.
I have: