• Lucene根据字段进行自定义搜索扩展


    最近需要对公司的产品搜索功能做一步改动,搜索到的结果首先按照是否有库存进行排序,然后再按照销量。由于库存量也是一个整数,如果直接按照库存量进行倒序排序的话,是不符合要求的,Lucene也没有支持我们这种特殊的业务需求,但是可以通过扩展的方式进行改写。
     
     
    public class EmptyStockComparatorSource extends FieldComparatorSource {
        @Override
        public FieldComparator<?> newComparator(String fieldname, int numHits, int sortPos, boolean reversed)
                throws IOException {
            return new LongComparator(numHits, fieldname, 0L);
        }
    
        public static class LongComparator extends FieldComparator.NumericComparator<Long> {
            private final long[] values;
            private long bottom;
            private long topValue;
    
            /**
             * Creates a new comparator based on {@link Long#compare} for {@code numHits}.
             * When a document has no value for the field, {@code missingValue} is substituted.
             */
            public LongComparator(int numHits, String field, Long missingValue) {
                super(field, missingValue);
                values = new long[numHits];
            }
    
            @Override
            protected void doSetNextReader(LeafReaderContext context) throws IOException {
                currentReaderValues = getNumericDocValues(context, field);
                if (missingValue != null) {
                    docsWithField = getDocsWithValue(context, field);
                    // optimization to remove unneeded checks on the bit interface:
                    if (docsWithField instanceof Bits.MatchAllBits) {
                        docsWithField = null;
                    }
                } else {
                    docsWithField = null;
                }
            }
    
            @Override
            public int compare(int slot1, int slot2) {
                return Long.compare(values[slot1], values[slot2]);
            }
    
            @Override
            public int compareBottom(int doc) {
                // TODO: there are sneaky non-branch ways to compute
                // -1/+1/0 sign
                long v2 = currentReaderValues.get(doc);
                // Test for v2 == 0 to save Bits.get method call for
                // the common case (doc has value and value is non-zero):
                if (docsWithField != null && v2 == 0 && !docsWithField.get(doc)) {
                    v2 = missingValue;
                }
    
                return Long.compare(bottom, v2);
            }
    
            @Override
            public void copy(int slot, int doc) {
                long v2 = currentReaderValues.get(doc);
                // Test for v2 == 0 to save Bits.get method call for
                // the common case (doc has value and value is non-zero):
                if (docsWithField != null && v2 == 0 && !docsWithField.get(doc)) {
                    v2 = missingValue;
                }
    
                values[slot] = v2 > 0L ? 1L : 0L;
            }
    
            @Override
            public void setBottom(final int bottom) {
                this.bottom = values[bottom];
            }
    
            @Override
            public void setTopValue(Long value) {
                topValue = value;
            }
    
            @Override
            public Long value(int slot) {
                return Long.valueOf(values[slot]) ;
            }
    
            @Override
            public int compareTop(int doc) {
                long docValue = currentReaderValues.get(doc);
                // Test for docValue == 0 to save Bits.get method call for
                // the common case (doc has value and value is non-zero):
                if (docsWithField != null && docValue == 0 && !docsWithField.get(doc)) {
                    docValue = missingValue;
                }
                return Long.compare(topValue, docValue);
            }
        }
    }
     
     
    其中LongComparator直接从lucene源码中copy出来,只需要做些许修改即可,最主要的修改就是copy(int slot, int doc)方法,在复制比较值得过程中,将所有存在库存的值都视为1,否则视为0,这样排序的结果就是我们所期待的。
     
    我们用到的测试用例:
     
    Directory directory1 = FSDirectory.open(Paths.get(
                    "/Users/xxx/develop/tools/solr-5.5.0/server/solr/product/data/index"));
            DirectoryReader directoryReader1 = DirectoryReader.open(directory1);
            IndexSearcher searcher1 = new IndexSearcher(directoryReader1);
            Sort sort1 = new Sort(new SortField("psfixstock", new EmptyStockComparatorSource(), true),
                    new SortField("salesVolume", SortField.Type.INT, true));
    
            TopFieldDocs topDocs1 = searcher1.search(new TermQuery(new Term("gender_text", "女士")), 10, sort1);
            for (ScoreDoc scoreDoc : topDocs1.scoreDocs) {
                int doc = scoreDoc.doc;
                Document document = searcher1.doc(doc);
                System.out.println(String.format("docId=%s, psfixstock=%s, salesVolumn=%s", doc, document.get("psfixstock"), document.get("salesVolume")));
            }
     
     
    在排序时,需要将其加入至Sort对象中,但执行的时候出现错误,显示docvalues的类型不正确:
     
    Exception in thread "main" java.lang.IllegalStateException: unexpected docvalues type NONE for field 'psfixstock' (expected=NUMERIC). Use UninvertingReader or index with docvalues.
        at org.apache.lucene.index.DocValues.checkField(DocValues.java:208)
        at org.apache.lucene.index.DocValues.getNumeric(DocValues.java:227)
        at org.apache.lucene.search.FieldComparator$NumericComparator.getNumericDocValues(FieldComparator.java:167)
        at com.zp.solr.handler.component.EmptyStockComparatorSource$LongComparator.doSetNextReader(EmptyStockComparatorSource.java:36)
        at org.apache.lucene.search.SimpleFieldComparator.getLeafComparator(SimpleFieldComparator.java:36)
        at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
        at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:164)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:812)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:744)
        at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:729)
        at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:671)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:577)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:627)
        at com.zp.solr.handler.component.EmptyStockSortingTest.main(EmptyStockSortingTest.java:57)
    
     
    经过一番查找,找到原因,参考文档:http://qindongliang.iteye.com/blog/2297280,我们搜索所使用到的字段没有设置对应的docType。如果在solr中,需要进行手动排序的字段,设置docValues=“true”,并进行重新索引(使用full-import方式):
     
       
    <field name="psfixstock" type="tint" indexed="true" stored="true" multiValued="false" docValues="true" />
     
     
    必须要重新建立索引才可以正常运行。注意,此时Solr与Elastic Search采取的方案有所不同,Solr默认docValues=false,而ES则相反,使用Doc索引方式会对性能产生一定的影响,要谨慎使用。
     
    对于lucene中,需要将添加document中增加数字类型Field:NumericDocValuesField,否则出现上面的错误,
     
    document.add(new NumericDocValuesField("stock", stock));
    
     
     
    最终的排序结果已经按照我们的需要进行了:
     
    docId=2629, psfixstock=98391, salesVolumn=4685
    docId=305, psfixstock=991, salesVolumn=14
    docId=16762, psfixstock=3, salesVolumn=12
    docId=22350, psfixstock=993, salesVolumn=10
    docId=29021, psfixstock=11076, salesVolumn=10
    docId=3635, psfixstock=61, salesVolumn=6
    docId=4111, psfixstock=1104, salesVolumn=5
    docId=10608, psfixstock=4395, salesVolumn=5
    docId=4874, psfixstock=4975, salesVolumn=4
    docId=4911, psfixstock=6, salesVolumn=4
    docId=15071, psfixstock=998, salesVolumn=4
    docId=4837, psfixstock=9, salesVolumn=3
    docId=4860, psfixstock=1002, salesVolumn=3
    docId=3749, psfixstock=2240, salesVolumn=2
    docId=4109, psfixstock=1493, salesVolumn=2
    docId=15068, psfixstock=1000, salesVolumn=2
    docId=25901, psfixstock=11110, salesVolumn=2
    docId=3688, psfixstock=21, salesVolumn=1
    docId=4912, psfixstock=17, salesVolumn=1
    docId=5035, psfixstock=2, salesVolumn=1
    docId=11835, psfixstock=8, salesVolumn=1
    docId=12044, psfixstock=1, salesVolumn=1
    docId=13508, psfixstock=2, salesVolumn=1
    docId=20019, psfixstock=1, salesVolumn=1
    docId=20884, psfixstock=100000, salesVolumn=1
    docId=22620, psfixstock=1, salesVolumn=1
    docId=24128, psfixstock=1, salesVolumn=1
    docId=0, psfixstock=2, salesVolumn=0
    docId=9, psfixstock=1, salesVolumn=0
    docId=11, psfixstock=4, salesVolumn=0
    docId=15, psfixstock=3, salesVolumn=0
    docId=20, psfixstock=4, salesVolumn=0
    docId=23, psfixstock=2, salesVolumn=0
    docId=24, psfixstock=5, salesVolumn=0
    docId=25, psfixstock=7, salesVolumn=0
    docId=35, psfixstock=2, salesVolumn=0
    docId=53, psfixstock=2, salesVolumn=0
     
     
     
     
  • 相关阅读:
    部署单master节点的k8s集群
    NFS搭建
    华为交换机操作
    MariaDB Galera集群部署
    docker学习笔记
    《Java虚拟机》学习2-Java平台无关性
    《Java虚拟机》学习3-安全
    《Java虚拟机》学习1-Java体系结构介绍
    Office 2010 禁用加载项, 给启动提速
    用lodashes代替lodash
  • 原文地址:https://www.cnblogs.com/mmaa/p/5789862.html
Copyright © 2020-2023  润新知