Add bedrock feedback support #958

lrafeei · 2023-11-03T21:08:02Z

Overview

Modify get_ai_message_ids API function to retrieve message IDs to be used in the customer's AI feedback endpoint.

commit 2834663 Author: Timothy Pansino <[email protected]> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <[email protected]> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit 182c7a8 Author: Uma Annamalai <[email protected]> Date: Fri Oct 13 10:12:55 2023 -0700 Add request/ response IDs. commit f6d13f8 Author: Uma Annamalai <[email protected]> Date: Thu Oct 12 13:23:39 2023 -0700 Test cleanup. commit d057663 Author: Uma Annamalai <[email protected]> Date: Tue Oct 10 10:23:00 2023 -0700 Remove commented code. commit dd29433 Author: Uma Annamalai <[email protected]> Date: Tue Oct 10 10:19:01 2023 -0700 Add openai sync instrumentation. commit 2834663 Author: Timothy Pansino <[email protected]> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <[email protected]> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]>

* Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact

Co-authored-by: Lalleh Rafeei <[email protected]>

Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]>

… into feature-bedrock-sync-instrumentation

…instrumentation

github-actions · 2023-11-03T21:09:38Z

🦙 MegaLinter status: ❌ ERROR

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ PYTHON	bandit	3		0	4.94s
✅ PYTHON	black	5	2	0	1.25s
❌ PYTHON	flake8	5		1	0.49s
✅ PYTHON	isort	5	2	0	0.23s
✅ PYTHON	pylint	5		0	3.91s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by

codecov-commenter · 2023-11-06T20:35:51Z

Codecov Report

❗ No coverage uploaded for pull request base (develop-bedrock-instrumentation@0882ba4). Click here to learn what that means.
The diff coverage is n/a.

@@                        Coverage Diff                         @@
##             develop-bedrock-instrumentation     #958   +/-   ##
==================================================================
  Coverage                                   ?   81.90%           
==================================================================
  Files                                      ?      190           
  Lines                                      ?    19767           
  Branches                                   ?     3436           
==================================================================
  Hits                                       ?    16190           
  Misses                                     ?     2590           
  Partials                                   ?      987

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

hmstepanek

I had a couple changes. Primarily, we are missing the request_id in the tuple of ids returned when get_ai_message_ids is called.

I had some additional feedback about the tests to make sure we are covering everything.

newrelic/api/ml_model.py

newrelic/hooks/external_botocore.py

tests/external_botocore/test_get_ai_message_ids.py

hmstepanek

In Bedrock the customer is never required to provide the response_id to get the message ids so the requested changes are centered around that. The response_id is required for OpenAI since OpenAI has async capabilities and we need to be able to distinguish one call's message ids from another's (since you could have multiple async calls before calling get_ai_message_ids). This is not the case in Bedrock since it doesn't support async so the customer doesn't have to pass an identifier in for us to know which bedrock call they want message ids for.

We recently decided to change the name of this public api function to get_llm_message_ids so while you're at it if you wouldn't mind updating the name in your PR, Uma is taking care of the rest of the name changes in her PR to openai.

Otherwise everything else looks good.

newrelic/api/ml_model.py

newrelic/hooks/external_botocore.py

tests/external_botocore/test_bedrock_chat_completion.py

hmstepanek

Looks great!

umaannamalai and others added 27 commits October 13, 2023 11:18

Add AWS Bedrock testing infrastructure

159b500

Cache Package Version Lookups (#946)

cc3e285

* Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]>

TEMP

360899b

Bedrock titan extraction nearly complete

8ddd0f4

Cleaning up titan bedrock implementation

bdbfdd3

TEMP

05dbe2a

Tests for bedrock passing

c305566

Co-authored-by: Lalleh Rafeei <[email protected]>

Cleaned up titan testing

024b24b

Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]>

Parametrized bedrock testing

4bc91bf

Add support for AI21-J2 models

fbb5f4d

Change to dynamic no conversation id events

b8c063f

Drop all openai refs

a1e7732

[Mega-Linter] Apply linters fixes

a8d3f3e

Adding response_id and response_model

1707d6f

Drop python 3.7 tests for Hypercorn (#954)

2191684

Merge remote-tracking branch 'origin/develop-bedrock-instrumentation'…

73f098c

… into feature-bedrock-sync-instrumentation

Merge branch 'main' into feature-bedrock-sync-instrumentation

30020c6

Merge remote-tracking branch 'origin/main' into feature-bedrock-sync-…

cac0dc6

…instrumentation

Apply suggestions from code review

3b4cf9d

Remove unused import

97064e8

Initial feedback commit for botocore

bfc962b

Bedrock feedback w/ testing for titan and jurassic models

7cf63cc

Merge branch 'develop-bedrock-instrumentation' into add-bedrock-feedback

8acc9b2

lrafeei changed the base branch from main to develop-bedrock-instrumentation November 3, 2023 21:08

mergify bot added the tests-failing Tests failing in CI. label Nov 3, 2023

Fix merge conflicts

8336e15

mergify bot added the merge-conflicts Merge conflicts detected. label Nov 6, 2023

Merge branch 'develop-bedrock-instrumentation' into add-bedrock-feedback

2fe21f4

mergify bot removed the merge-conflicts Merge conflicts detected. label Nov 6, 2023

hmstepanek requested changes Nov 6, 2023

View reviewed changes

hmstepanek marked this pull request as ready for review November 6, 2023 23:10

hmstepanek requested a review from a team November 6, 2023 23:10

lrafeei and others added 2 commits November 8, 2023 12:16

Add to and move feedback tests

8a04de6

Handle 0.32.0.post1 version in tests (#963)

91ddbda

mergify bot removed the tests-failing Tests failing in CI. label Nov 8, 2023

lrafeei requested a review from hmstepanek November 8, 2023 20:43

hmstepanek requested changes Nov 9, 2023

View reviewed changes

lrafeei added 2 commits November 9, 2023 10:22

Remove response_id dependency in bedrock

f99c20b

Change API name

8800cfc

mergify bot added the tests-failing Tests failing in CI. label Nov 9, 2023

Update moto

de855c6

mergify bot added merge-conflicts Merge conflicts detected. and removed tests-failing Tests failing in CI. labels Nov 9, 2023

lrafeei requested a review from hmstepanek November 9, 2023 21:54

Merge branch 'develop-bedrock-instrumentation' into add-bedrock-feedback

4168ec9

mergify bot added tests-failing Tests failing in CI. and removed merge-conflicts Merge conflicts detected. labels Nov 9, 2023

Change ids to match other tests

af1c31e

mergify bot removed the tests-failing Tests failing in CI. label Nov 9, 2023

hmstepanek approved these changes Nov 10, 2023

View reviewed changes

hmstepanek merged commit ef2f39c into develop-bedrock-instrumentation Nov 10, 2023

hmstepanek deleted the add-bedrock-feedback branch November 10, 2023 00:07

lrafeei mentioned this pull request Jan 9, 2024

Add bedrock feedback into preview #1030

Merged

umaannamalai added this to the v9.8.0 milestone Mar 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add bedrock feedback support #958

Add bedrock feedback support #958

Uh oh!

lrafeei commented Nov 3, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Nov 3, 2023 •

edited

Loading

Uh oh!

codecov-commenter commented Nov 6, 2023 •

edited

Loading

Uh oh!

hmstepanek left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmstepanek left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmstepanek left a comment

Uh oh!

Uh oh!

Add bedrock feedback support #958

Add bedrock feedback support #958

Uh oh!

Conversation

lrafeei commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Uh oh!

github-actions bot commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦙 MegaLinter status: ❌ ERROR

Uh oh!

codecov-commenter commented Nov 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

hmstepanek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmstepanek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmstepanek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lrafeei commented Nov 3, 2023 •

edited

Loading

github-actions bot commented Nov 3, 2023 •

edited

Loading

codecov-commenter commented Nov 6, 2023 •

edited

Loading