Skip to content

Bug fixes and refactoring - #38

Merged
macohen merged 3 commits into
opensearch-project:mainfrom
msfroh:generic-refactoring
Dec 19, 2022
Merged

Bug fixes and refactoring#38
macohen merged 3 commits into
opensearch-project:mainfrom
msfroh:generic-refactoring

Conversation

@msfroh

@msfroh msfroh commented Dec 2, 2022

Copy link
Copy Markdown
Collaborator
  1. Shallow clone SearchResponse in SearchActionFilter. The only properties that we're currently changing are SearchHit and timeTookMillis. This is also less brittle as we don't try to implement deserialization ourselves.
  2. KendraIntelligentRanker doesn't transform if the required client settings (endpoint + execution plan ID) are missing.
  3. A lot of refactoring to prepare for other transformers. Where we previously had logic of the form if kendra_intelligent_ranking then do Kendra intelligent ranking, now the APIs support arbitrary named transformers. This required updates to tests and configuration code.
  4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh froh@amazon.com

Description

See commit message above.

Issues Resolved

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@msfroh
msfroh requested a review from a team December 2, 2022 19:46
@macohen
macohen requested a review from noCharger December 14, 2022 04:15

@noCharger noCharger left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess setting class is more specific towards that transformer itself, whereas the configuration is the schema of the search workflow. There are two things I am not sure if we should put in this PR:

  1. Can I use multiple same reranker, with different setting and configuration?
  2. Do we want to provide more flexibility, such as one transformer depend on several other transformers.

1. Shallow clone SearchResponse in SearchActionFilter. The only
   properties that we're currently changing are SearchHit and
   timeTookMillis. This is also less brittle as we don't try to
   implement deserialization ourselves.
2. KendraIntelligentRanker doesn't transform if the required client
   settings (endpoint + execution plan ID) are missing.
3. A lot of refactoring to prepare for other transformers. Where we
   previously had logic of the form `if kendra_intelligent_ranking then
   do Kendra intelligent ranking`, now the APIs support arbitrary named
   transformers. This required updates to tests and configuration code.
4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>
Looking at the PR on GitHub, I spotted a few mistakes and possible
improvements.

Signed-off-by: Michael Froh <froh@amazon.com>
Signed-off-by: Michael Froh <froh@amazon.com>
@msfroh
msfroh force-pushed the generic-refactoring branch from 2a40bd3 to 698f275 Compare December 17, 2022 01:13
@codecov-commenter

codecov-commenter commented Dec 17, 2022

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.60265% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.52%. Comparing base (d9b44e7) to head (698f275).
⚠️ Report is 63 commits behind head on main.

Files with missing lines Patch % Lines
...draintelligentranking/client/KendraHttpClient.java 23.33% 21 Missing and 2 partials ⚠️
...e/configuration/SearchConfigurationExtBuilder.java 66.66% 12 Missing and 10 partials ⚠️
...search/search/relevance/SearchRelevancePlugin.java 0.00% 6 Missing ⚠️
.../KendraIntelligentRankingConfigurationFactory.java 0.00% 5 Missing ⚠️
...ch/relevance/configuration/ConfigurationUtils.java 60.00% 1 Missing and 1 partial ⚠️
...rch/relevance/actionfilter/SearchActionFilter.java 96.00% 1 Missing ⚠️
...draintelligentranking/KendraIntelligentRanker.java 91.66% 1 Missing ⚠️
...uration/KendraIntelligentRankingConfiguration.java 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main      #38      +/-   ##
============================================
+ Coverage     60.43%   65.52%   +5.09%     
- Complexity      155      172      +17     
============================================
  Files            29       29              
  Lines           867      879      +12     
  Branches        119      119              
============================================
+ Hits            524      576      +52     
+ Misses          304      254      -50     
- Partials         39       49      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

objectMapper = new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
serviceEndpoint = clientSettings.getServiceEndpoint();
executionPlanId = clientSettings.getExecutionPlanId();
if (isValid()) {

@noCharger noCharger Dec 19, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking:

This is what I preferred:

    if (!isValid()) {
      amazonHttpClient = null;
      aws4Signer = null;
      awsCredentialsProvider = null;
      errorHandler = null;
      responseHandler = null;
      return;
    }
    amazonHttpClient = AccessController.doPrivileged((PrivilegedAction<AmazonHttpClient>) () -> new AmazonHttpClient(new ClientConfiguration()));
    errorHandler = new SimpleAwsErrorHandler();
    responseHandler = new SimpleResponseHandler();
    aws4Signer = new AWS4Signer();
    aws4Signer.setServiceName(KENDRA_RANKING_SERVICE_NAME);
    aws4Signer.setRegionName(clientSettings.getServiceRegion());
   ...

@noCharger noCharger left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome! Approved with non-blocking comments.

@macohen

macohen commented Dec 19, 2022

Copy link
Copy Markdown
Collaborator

We do want to review the full set of metrics and logging events to capture. We'll create another issue for that purpose.

@macohen
macohen merged commit d8ba75b into opensearch-project:main Dec 19, 2022
@noCharger noCharger added backport 2.x Backport to 2.x branch enhancement change or upgrade that increases software capabilities beyond original client specifications labels Dec 20, 2022
@noCharger noCharger added backport 2.x Backport to 2.x branch and removed backport 2.x Backport to 2.x branch labels Jan 3, 2023
opensearch-trigger-bot Bot pushed a commit that referenced this pull request Jan 3, 2023
* Bug fixes and refactoring

1. Shallow clone SearchResponse in SearchActionFilter. The only
   properties that we're currently changing are SearchHit and
   timeTookMillis. This is also less brittle as we don't try to
   implement deserialization ourselves.
2. KendraIntelligentRanker doesn't transform if the required client
   settings (endpoint + execution plan ID) are missing.
3. A lot of refactoring to prepare for other transformers. Where we
   previously had logic of the form `if kendra_intelligent_ranking then
   do Kendra intelligent ranking`, now the APIs support arbitrary named
   transformers. This required updates to tests and configuration code.
4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address PR comments from myself

Looking at the PR on GitHub, I spotted a few mistakes and possible
improvements.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address comments by noCharger

Signed-off-by: Michael Froh <froh@amazon.com>

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit d8ba75b)
noCharger pushed a commit that referenced this pull request Jan 3, 2023
* Bug fixes and refactoring

1. Shallow clone SearchResponse in SearchActionFilter. The only
   properties that we're currently changing are SearchHit and
   timeTookMillis. This is also less brittle as we don't try to
   implement deserialization ourselves.
2. KendraIntelligentRanker doesn't transform if the required client
   settings (endpoint + execution plan ID) are missing.
3. A lot of refactoring to prepare for other transformers. Where we
   previously had logic of the form `if kendra_intelligent_ranking then
   do Kendra intelligent ranking`, now the APIs support arbitrary named
   transformers. This required updates to tests and configuration code.
4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address PR comments from myself

Looking at the PR on GitHub, I spotted a few mistakes and possible
improvements.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address comments by noCharger

Signed-off-by: Michael Froh <froh@amazon.com>

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit d8ba75b)

Co-authored-by: msfroh <froh@amazon.com>
@mingshl mingshl added bug Something isn't working and removed enhancement change or upgrade that increases software capabilities beyond original client specifications labels Jan 23, 2023
@mingshl mingshl mentioned this pull request Jan 26, 2023
5 tasks
@mingshl mingshl added maintenance A change to add support for new versions of OpenSearch or OpenSearch Dashboards from upstream. enhancement change or upgrade that increases software capabilities beyond original client specifications and removed bug Something isn't working maintenance A change to add support for new versions of OpenSearch or OpenSearch Dashboards from upstream. labels Jan 26, 2023
@msfroh
msfroh deleted the generic-refactoring branch February 7, 2023 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.x Backport to 2.x branch enhancement change or upgrade that increases software capabilities beyond original client specifications

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants