Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6645

BKD tree queries should use BitDocIdSet.Builder

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 5.3, 6.0
    • None
    • None
    • New

    Description

      When I was iterating on BKD tree originally I remember trying to use this builder (which makes a sparse bit set at first and then upgrades to dense if enough bits get set) and being disappointed with its performance.

      I wound up just making a FixedBitSet every time, but this is obviously wasteful for small queries.

      It could be the perf was poor because I was always .or'ing in DISIs that had 512 - 1024 hits each time (the size of each leaf cell in the BKD tree)? I also had to make my own DISI wrapper around each leaf cell... maybe that was the source of the slowness, not sure.

      I also sort of wondered whether the SmallDocSet in spatial module (backed by a SentinelIntSet) might be faster ... though it'd need to be sorted in the and after building before returning to Lucene.

      Attachments

        1. LUCENE-6645.patch
          4 kB
          Michael McCandless
        2. LUCENE-6645.patch
          3 kB
          Michael McCandless
        3. LUCENE-6645.patch
          59 kB
          Adrien Grand
        4. LUCENE-6645.patch
          51 kB
          Adrien Grand
        5. LUCENE-6645.patch
          25 kB
          Adrien Grand
        6. LUCENE-6645.patch
          13 kB
          Michael McCandless
        7. LUCENE-6645-spatial.patch
          10 kB
          Adrien Grand

        Activity

          People

            Unassigned Unassigned
            mikemccand Michael McCandless
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: