Skip to content

Merge all changes to 2.x branch before cutting 2.4 branch - #34

Merged
msfroh merged 18 commits into
2.xfrom
main
Nov 23, 2022
Merged

Merge all changes to 2.x branch before cutting 2.4 branch#34
msfroh merged 18 commits into
2.xfrom
main

Conversation

@msfroh

@msfroh msfroh commented Nov 23, 2022

Copy link
Copy Markdown
Collaborator

This PR is just a backport of all code to 2.x before we cut a 2.4 branch from 2.x.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Michael Froh froh@amazon.com

macohen and others added 18 commits November 8, 2022 10:05
* adding .github workflows

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* * moved source for semantic-ranker plugin to root to follow 1 plugin per repo convention
* moved README.md to RELEVANCE.md; will create new README for search request processor, current, and future proposed states

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* deleted semantic-reranker dir after moving to root

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* added msfroh, ps48, yang-db, and macohen to maintainers

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* commented qa workflow build; force added gradle-wrapper.jar after gitignore ignored it

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* adding release notes folder

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* commented windows build to get the changes into a PR; filing windows build issue #19

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* commented qa gradle build for BWC; need to add it back and work on missing qa folder and what should be in there

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* Modified README to reflect current state of plugin hosted in the repo.

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* commented out more windows build just to get macos and ubuntu workflows working; replaced k-NN reference

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* missed a line to comment

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* cleaning up documentation

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* deleting editor swap files

Signed-off-by: Mark Cohen <markcoh@amazon.com>

Signed-off-by: Mark Cohen <markcoh@amazon.com>
Signed-off-by: Mahita Mahesh <mahitam@amazon.com>

Signed-off-by: Mahita Mahesh <mahitam@amazon.com>
Co-authored-by: Mahita Mahesh <mahitam@amazon.com>
* Update gitignore for Intellij

Signed-off-by: Mahita Mahesh <mahitam@amazon.com>

Signed-off-by: Mahita Mahesh <mahitam@amazon.com>
Co-authored-by: Mahita Mahesh <mahitam@amazon.com>
As we get closer to releasing this plugin to work with 2.4, we should
update the name -- going with search-processor -- to better reflect the
idea that it will be able to transform search requests and responses.
Also, we need to update the build to point to 2.4 snapshots, instead of
2.1.

Signed-off-by: Michael Froh <froh@amazon.com>
Change plugin name and depend on 2.4
Since 2.4.0 is released, we no longer need to build against -SNAPSHOT.

Signed-off-by: Michael Froh <froh@amazon.com>
Build against 2.4.0 (without -SNAPSHOT)
* adding codecov and windows build commands
* output jacoco test reports
* enabling jacoco test report output in build.gradle
* adding dependabot check at a daily interval
* added test that adds 1 more line of coverage. :|
* fixing dependabot configurations
* fixing tests - adding 1 line coverage - woohoo
* removed spotlessCheck

Signed-off-by: Mark Cohen <markcoh@amazon.com>
Signed-off-by: Michael Froh <froh@amazon.com>
…than the default 10.

* Add preprocess request interface to ResultTransformer
* Override from and size parameters durig preprocess, to fetch more hits from OpenSearch
* Limit the number of documents sent to Kendra Ranking (Will make this number configurable in a follow up)
* Populate response documents from re-ranked hits (upto Kendra limit) and OpenSearch hits beyond the limit

Signed-off-by: Mahita Mahesh <mahitam@amazon.com>
Signed-off-by: Michael Froh <froh@amazon.com>
Signed-off-by: Mahita Mahesh <mahitam@amazon.com>
Signed-off-by: Mahita Mahesh <mahitam@amazon.com>
Signed-off-by: Michael Froh <froh@amazon.com>
Add unit tests for the pagination logic in SearchActionFilter as well
as the whole KendraIntelligentRanker class (including pagination).

Also, removed OriginalRequestFields in favor of cloning the
SearchSourceBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>
Updating integration test to match approach used by other recent
plugins, in the hope of fixing test runs on Windows.

Signed-off-by: Michael Froh <froh@amazon.com>
* Fix test case with non-null source but null query.
* Skip transform on empty results before doing any work.

Signed-off-by: Michael Froh <froh@amazon.com>
* Earlier test covered empty hits, so we should assume that Kendra
  ranking handles at least 1 hit.

Signed-off-by: Michael Froh <froh@amazon.com>
Add pagination / document limit logic for reranking
Signed-off-by: Mahita Mahesh <mahitam@amazon.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❗ No coverage uploaded for pull request base (2.x@9c3994b). Click here to learn what that means.
The diff coverage is n/a.

@@          Coverage Diff           @@
##             2.x      #34   +/-   ##
======================================
  Coverage       ?   60.43%           
  Complexity     ?      155           
======================================
  Files          ?       29           
  Lines          ?      867           
  Branches       ?      119           
======================================
  Hits           ?      524           
  Misses         ?      304           
  Partials       ?       39           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@msfroh
msfroh merged commit 1cbc376 into 2.x Nov 23, 2022
@mingshl mingshl added the maintenance A change to add support for new versions of OpenSearch or OpenSearch Dashboards from upstream. label Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance A change to add support for new versions of OpenSearch or OpenSearch Dashboards from upstream.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants