Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test] Refactored test to use utility function to get annotation instead of array indexing #1149

Merged
merged 1 commit into from
Nov 3, 2023

Conversation

wang3820
Copy link

@wang3820 wang3820 commented Nov 2, 2023

Following this checklist to help us incorporate your contribution quickly and easily:

  • Make sure there is a GitHub issue filed
    for the change (usually before you start working on it). Trivial changes like typos do not
    require a GitHub issue. Your pull request should address just this issue, without pulling in other changes.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [#XXX] - Fixes bug in SessionManager,
    where you replace #XXX with the appropriate GitHub issue. Best practice
    is to use the GitHub issue title in the pull request title and in the first line of the commit message.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • add fixes #XXX if merging the PR should close a related issue.
  • Run mvn verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
  • Committers: Make sure a milestone is set on the PR

Trivial changes like typos do not require a GitHub issue (javadoc, comments...).
In this case, just format the pull request title like [DOC] - Add javadoc in SessionManager.

If this is your first contribution, you have to read the Contribution Guidelines

If your pull request is about ~20 lines of code you don't need to sign an Individual Contributor License Agreement
if you are unsure please ask on the developers list.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

Flakiness

Nondex was used to check and locate the flakiness in the test. The test can be reproduced using the following command:

mvn install -pl support/cdi -am -DskipTests
mvn -pl support/cdi test -Dtest=org.apache.shiro.cdi.AnnotatedTypeWrapperTest#removeAnnotations
mvn -pl support/cdi edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.shiro.cdi.AnnotatedTypeWrapperTest#removeAnnotations

Issue

The test checks that if there is no Annotation in the RequiresGuest class and there is Annotation in SessionScoped class. It retrieves these classes by calling getDeclaredAnnotations method and fix indexing on the array returned. This creates the problem. Here is the implementation of getDeclaredAnnotations :

public Annotation[] getDeclaredAnnotations() {
     return AnnotationParser.toArray(this.annotationData().declaredAnnotations);
}
// this.annotationData().declaredAnnotationsdeclaredAnnotations is a map
Map<Class<? extends Annotation>, Annotation> declaredAnnotations = ....

Since the array is created from a hashmap, the order of elements in the output of getDeclaredAnnotations is not determinstic. Therefore, the fix indexing creates flakiness.

Fix

Since only SessionScoped and RequiresGuest are tested, those 2 classes are specificly extracted from the array instead of using indexing.

@lprimak lprimak requested a review from bdemers November 2, 2023 19:52
lprimak
lprimak previously approved these changes Nov 2, 2023
@lprimak lprimak added this to the 2.0 milestone Nov 2, 2023
@lprimak
Copy link
Contributor

lprimak commented Nov 2, 2023

@wang3820 Can you please squash the commits? Thank you!

@lprimak
Copy link
Contributor

lprimak commented Nov 2, 2023

@wang3820 I have looked a little deeper into this. I believe it's a false positive actually.
The Map involved is actually an instance of LinkedHashMap so the order will never change.

Let me know if you still want me to merge this PR or just close it.
Thank you.

@lprimak lprimak dismissed their stale review November 2, 2023 20:03

this may not be necessary after all

@lprimak lprimak self-requested a review November 2, 2023 20:03
@wang3820
Copy link
Author

wang3820 commented Nov 2, 2023

@wang3820 I have looked a little deeper into this. I believe it's a false positive actually. The Map involved is actually an instance of LinkedHashMap so the order will never change.

Let me know if you still want me to merge this PR or just close it. Thank you.

Let me look deeper into this too. Nondex should never shuffle order of elements in a LinkedHashMap

@bdemers
Copy link
Member

bdemers commented Nov 2, 2023

Let me look deeper into this too. Nondex should never shuffle order of elements in a LinkedHashMap

Even so, your change makes it obvious what is going on (e.g. looking for a specific annotation class vs just grabbing one by index)

@lprimak
Copy link
Contributor

lprimak commented Nov 2, 2023

The other issue is that the test uses indexes in the whole code, not just this piece. Do we want to change the whole test to be consistent?

@wang3820
Copy link
Author

wang3820 commented Nov 2, 2023

@wang3820 I have looked a little deeper into this. I believe it's a false positive actually. The Map involved is actually an instance of LinkedHashMap so the order will never change.
Let me know if you still want me to merge this PR or just close it. Thank you.

Let me look deeper into this too. Nondex should never shuffle order of elements in a LinkedHashMap

It appears that the test is no longer flaky. I think I was testing on a older version.

@wang3820
Copy link
Author

wang3820 commented Nov 2, 2023

The other issue is that the test uses indexes in the whole code, not just this piece. Do we want to change the whole test to be consistent?

I think it's a good idea to change the whole test. This way the test is more readable.

@lprimak
Copy link
Contributor

lprimak commented Nov 2, 2023

@wang3820 are you willing to take that on?

@wang3820
Copy link
Author

wang3820 commented Nov 2, 2023

@wang3820 are you willing to take that on?

Yes.

@wang3820 wang3820 requested a review from bdemers November 2, 2023 23:44
@wang3820 wang3820 changed the title [Flaky] - Fixed flaky testing in cdi/AnnotatedTypeWrapperTest Nov 2, 2023
@wang3820
Copy link
Author

wang3820 commented Nov 2, 2023

@bdemers @lprimak I have changed all the tests so it is consistent.

I am rather new to this so I would really appreciate any suggestions!

@lprimak
Copy link
Contributor

lprimak commented Nov 3, 2023

@wang3820 Awesome job! Thank you!
I will squash the commits and merge

@lprimak lprimak merged commit 9888e87 into apache:main Nov 3, 2023
2 of 3 checks passed
@bdemers
Copy link
Member

bdemers commented Nov 3, 2023

Thanks @wang3820!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants