Elasticsearch injini yekutsvaga ine json rest api, inoshandisa Lucene uye yakanyorwa muJava. Tsanangudzo yezvese zvakanakira injini iyi inowanikwa pa website yepamutemo. Mune zvinotevera tichareva Elasticsearch se ES.

Injini dzakafanana dzinoshandiswa kutsvaga kwakaoma mudura regwaro. Semuenzaniso, tsvaga uchifunga nezve morphology yemutauro kana tsvaga ne geo coordinates.

Muchinyorwa chino ini ndichataura nezve izvo zvekutanga zveES uchishandisa muenzaniso we indexing blog posts. Ini ndichakuratidza maitiro ekusefa, kuronga uye kutsvaga magwaro.

Kuti ndirege kutsamira pane inoshanda sisitimu, ini ndichaita zvikumbiro zvese kuES ndichishandisa CURL. Kune zvakare plugin yegoogle chrome inonzi pfungwa.

Chinyorwa chine zvinongedzo kune zvinyorwa uye mamwe masosi. Pakupedzisira kune zvinongedzo zvekukurumidza kuwana zvinyorwa. Tsanangudzo dzemazwi asina kujairika dzinogona kuwanikwa mukati glossaries.

Installation

Kuti tiite izvi, tinoda kutanga Java. Developers kurudzira isa Java vhezheni nyowani pane Java 8 update 20 kana Java 7 update 55.

Kugoverwa kweES kunowanikwa pa developer webhusaiti. Mushure mekusunungura iyo archive iwe unofanirwa kumhanya bin/elasticsearch. Zvakare zviripo mapakeji e apt uye yum. pane mufananidzo wepamutemo we docker. Zvimwe pamusoro pekuisa.

Mushure mekuisa uye kutanga, ngatitarisei kushanda:

# для удобства запомним адрес в переменную
#export ES_URL=$(docker-machine ip dev):9200
export ES_URL=localhost:9200

curl -X GET $ES_URL

Tichagamuchira chinhu chakadai:

{
  "name" : "Heimdall",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.2.1",
    "build_hash" : "d045fc29d1932bce18b2e65ab8b297fbf6cd41a1",
    "build_timestamp" : "2016-03-09T09:38:54Z",
    "build_snapshot" : false,
    "lucene_version" : "5.4.1"
  },
  "tagline" : "You Know, for Search"
}

Indexing

Ngatiwedzerei chinyorwa kuES:

# Добавим документ c id 1 типа post в индекс blog.
# ?pretty указывает, что вывод должен быть человеко-читаемым.

curl -XPUT "$ES_URL/blog/post/1?pretty" -d'
{
  "title": "Веселые котята",
  "content": "<p>Смешная история про котят<p>",
  "tags": [
    "котята",
    "смешная история"
  ],
  "published_at": "2014-09-12T20:44:42+00:00"
}'

server mhinduro:

{
  "_index" : "blog",
  "_type" : "post",
  "_id" : "1",
  "_version" : 1,
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "created" : false
}

ES yakagadzirwa otomatiki index blog uye mhando post. Tinogona kudhirowa kuenzanisa kwezvimiso: index idatabase, uye mhando itafura mune ino database. Mhando yega yega ine chirongwa chayo - nzvimbo ine, kungofanana netafura yehukama. Mepu inogadzirwa otomatiki kana gwaro raiswa indexed:

# Получим mapping всех типов индекса blog
curl -XGET "$ES_URL/blog/_mapping?pretty"

Mumhinduro yeseva, ini ndakawedzera kukosha kweminda yegwaro rakanyoreswa mune zvakataurwa:

{
  "blog" : {
    "mappings" : {
      "post" : {
        "properties" : {
          /* "content": "<p>Смешная история про котят<p>", */ 
          "content" : {
            "type" : "string"
          },
          /* "published_at": "2014-09-12T20:44:42+00:00" */
          "published_at" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          /* "tags": ["котята", "смешная история"] */
          "tags" : {
            "type" : "string"
          },
          /*  "title": "Веселые котята" */
          "title" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

Zvakakosha kucherechedza kuti ES haisiyanisi pakati pehukoshi humwechete uye huwandu hwehuwandu. Semuenzaniso, ndima yemusoro inongove nemusoro, uye iyo tags ndima ine tambo dzetambo, kunyangwe ivo vachimiririrwa nenzira imwechete mumepu.
Tichazotaura zvakawanda nezve mepu gare gare.

Zvikumbiro

Kutora gwaro neid yaro:

# извлечем документ с id 1 типа post из индекса blog
curl -XGET "$ES_URL/blog/post/1?pretty"

{
  "_index" : "blog",
  "_type" : "post",
  "_id" : "1",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "title" : "Веселые котята",
    "content" : "<p>Смешная история про котят<p>",
    "tags" : [ "котята", "смешная история" ],
    "published_at" : "2014-09-12T20:44:42+00:00"
  }
}

Makiyi matsva akaonekwa mumhinduro: _version и _source. Kazhinji, makiyi ese kutanga _ vanoiswa muboka rehurumende.

Key _version inoratidza vhezheni yegwaro. Inodiwa kuti nzira yekuvhara ine tariro ishande. Semuenzaniso, tinoda kushandura gwaro rine vhezheni 1. Tinoendesa gwaro rakashandurwa uye tinoratidza kuti iyi igadziriso yegwaro rine vhezheni 1. Kana mumwe munhu akagadzirisawo gwaro rine vhezheni 1 uye akaisa shanduko pamberi pedu, saka ES haizogamuchire shanduko dzedu, nekuti inochengeta gwaro neshanduro 2.

Key _source rine gwaro ratakaisa indexed. ES haishandise kukosha uku pakutsvaga mabasa nekuti Indexes inoshandiswa pakutsvaga. Kuti uchengetedze nzvimbo, ES inochengetedza gwaro rakadzvanywa sosi. Kana isu tichingoda id chete, uye kwete iyo yese sosi gwaro, saka tinogona kudzima sosi yekuchengetedza.

Kana isu tisingade rumwe ruzivo, tinogona kuwana chete zviri mukati _source:

curl -XGET "$ES_URL/blog/post/1/_source?pretty"

{
  "title" : "Веселые котята",
  "content" : "<p>Смешная история про котят<p>",
  "tags" : [ "котята", "смешная история" ],
  "published_at" : "2014-09-12T20:44:42+00:00"
}

Iwe unogona zvakare kusarudza mamwe minda chete:

# извлечем только поле title
curl -XGET "$ES_URL/blog/post/1?_source=title&pretty"

{
  "_index" : "blog",
  "_type" : "post",
  "_id" : "1",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "title" : "Веселые котята"
  }
}

Ngatinyorei mamwe mashoma uye tibvunze mibvunzo yakaoma.

curl -XPUT "$ES_URL/blog/post/2" -d'
{
  "title": "Веселые щенки",
  "content": "<p>Смешная история про щенков<p>",
  "tags": [
    "щенки",
    "смешная история"
  ],
  "published_at": "2014-08-12T20:44:42+00:00"
}'

curl -XPUT "$ES_URL/blog/post/3" -d'
{
  "title": "Как у меня появился котенок",
  "content": "<p>Душераздирающая история про бедного котенка с улицы<p>",
  "tags": [
    "котята"
  ],
  "published_at": "2014-07-21T20:44:42+00:00"
}'

Kugadziriswa

# найдем последний пост по дате публикации и извлечем поля title и published_at
curl -XGET "$ES_URL/blog/post/_search?pretty" -d'
{
  "size": 1,
  "_source": ["title", "published_at"],
  "sort": [{"published_at": "desc"}]
}'

{
  "took" : 8,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : null,
    "hits" : [ {
      "_index" : "blog",
      "_type" : "post",
      "_id" : "1",
      "_score" : null,
      "_source" : {
        "title" : "Веселые котята",
        "published_at" : "2014-09-12T20:44:42+00:00"
      },
      "sort" : [ 1410554682000 ]
    } ]
  }
}

Takasarudza positi yekupedzisira. size inoganhura nhamba yezvinyorwa zvinopihwa. total inoratidza nhamba yese yemagwaro anoenderana nechikumbiro. sort mune zvinobuda mune rondedzero yetigers iyo kurongwa kunoitwa. Avo. zuva rakashandurwa kuita nhamba yakazara. Rumwe ruzivo nezve kurongedza runogona kuwanikwa mukati zvinyorwa.

Sefa uye mibvunzo

ES sezvo vhezheni 2 isingasiyanise pakati pemafirita nemibvunzo, pane kudaro pfungwa yezvimiro inounzwa.
Chirevo chemubvunzo chinosiyana nesefa yemamiriro ekuti muvhunzo unoburitsa _score uye hauna kuchengetwa. Ndichakuratidza kuti _score ndezvipi.

Sefa nemazuva

Isu tinoshandisa chikumbiro dungwerungwe muchirevo chesefa:

# получим посты, опубликованные 1ого сентября или позже
curl -XGET "$ES_URL/blog/post/_search?pretty" -d'
{
  "filter": {
    "range": {
      "published_at": { "gte": "2014-09-01" }
    }
  }
}'

Sefa nema tags

Isu tinoshandisa term query kutsvaga ma ID egwaro rine izwi rakapihwa:

# найдем все документы, в поле tags которых есть элемент 'котята'
curl -XGET "$ES_URL/blog/post/_search?pretty" -d'
{
  "_source": [
    "title",
    "tags"
  ],
  "filter": {
    "term": {
      "tags": "котята"
    }
  }
}'

{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "blog",
      "_type" : "post",
      "_id" : "1",
      "_score" : 1.0,
      "_source" : {
        "title" : "Веселые котята",
        "tags" : [ "котята", "смешная история" ]
      }
    }, {
      "_index" : "blog",
      "_type" : "post",
      "_id" : "3",
      "_score" : 1.0,
      "_source" : {
        "title" : "Как у меня появился котенок",
        "tags" : [ "котята" ]
      }
    } ]
  }
}

Tsvakiridzo yemavara akazara

Matatu emagwaro edu ane zvinotevera mundima yemukati:

• <p>Смешная история про котят<p>
• <p>Смешная история про щенков<p>
• <p>Душераздирающая история про бедного котенка с улицы<p>

Isu tinoshandisa match mubvunzo kutsvaga ma ID egwaro rine izwi rakapihwa:

# source: false означает, что не нужно извлекать _source найденных документов
curl -XGET "$ES_URL/blog/post/_search?pretty" -d'
{
  "_source": false,
  "query": {
    "match": {
      "content": "история"
    }
  }
}'

{
  "took" : 13,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.11506981,
    "hits" : [ {
      "_index" : "blog",
      "_type" : "post",
      "_id" : "2",
      "_score" : 0.11506981
    }, {
      "_index" : "blog",
      "_type" : "post",
      "_id" : "1",
      "_score" : 0.11506981
    }, {
      "_index" : "blog",
      "_type" : "post",
      "_id" : "3",
      "_score" : 0.095891505
    } ]
  }
}

Nekudaro, kana tikatsvaga "nyaya" mundima yemukati, isu hatizowana chero chinhu, nekuti Indekisi ine mazwi ekutanga chete, kwete madzinde awo. Kuti uite kutsvaga kwepamusoro-soro, unoda kugadzirisa analyzer.

munda _score inoratidza kukosha. Kana chikumbiro chikaitwa musefa, ipapo kukosha kwe _score kunogara kwakaenzana ne1, izvo zvinoreva kuenzanisa kwakakwana kune sefa.

Analyzers

Analyzers inodiwa kushandura chinyorwa chekwakabva kuita seti yematokeni.
Analyzers anosanganisira imwe Tokenizer uye akati wandei sarudzo TokenFilters. Tokenizer inogona kutungamirwa nevanoverengeka CharFilters. Tokenizers inotyora tambo yekutangira kuita tokens, senge nzvimbo uye mavara epunctuation. TokenFilter inogona kushandura tokeni, kudzima kana kuwedzera matsva, semuenzaniso, siya chete hunde reshoko, bvisa prepositions, wedzera masinonimi. CharFilter - inoshandura iyo yese sosi tambo, semuenzaniso, inocheka ma tag ehtml.

ES ine akati wandei standard analyzers. Somuenzaniso, analyzer Russian.

Ngatitore mukana API uye ngationei kuti chiyero uye russian analyzers inoshandura sei tambo "Nyaya dzinosetsa nezvekitsi":

# используем анализатор standard       
# обязательно нужно перекодировать не ASCII символы
curl -XGET "$ES_URL/_analyze?pretty&analyzer=standard&text=%D0%92%D0%B5%D1%81%D0%B5%D0%BB%D1%8B%D0%B5%20%D0%B8%D1%81%D1%82%D0%BE%D1%80%D0%B8%D0%B8%20%D0%BF%D1%80%D0%BE%20%D0%BA%D0%BE%D1%82%D1%8F%D1%82"

{
  "tokens" : [ {
    "token" : "веселые",
    "start_offset" : 0,
    "end_offset" : 7,
    "type" : "<ALPHANUM>",
    "position" : 0
  }, {
    "token" : "истории",
    "start_offset" : 8,
    "end_offset" : 15,
    "type" : "<ALPHANUM>",
    "position" : 1
  }, {
    "token" : "про",
    "start_offset" : 16,
    "end_offset" : 19,
    "type" : "<ALPHANUM>",
    "position" : 2
  }, {
    "token" : "котят",
    "start_offset" : 20,
    "end_offset" : 25,
    "type" : "<ALPHANUM>",
    "position" : 3
  } ]
}

# используем анализатор russian
curl -XGET "$ES_URL/_analyze?pretty&analyzer=russian&text=%D0%92%D0%B5%D1%81%D0%B5%D0%BB%D1%8B%D0%B5%20%D0%B8%D1%81%D1%82%D0%BE%D1%80%D0%B8%D0%B8%20%D0%BF%D1%80%D0%BE%20%D0%BA%D0%BE%D1%82%D1%8F%D1%82"

{
  "tokens" : [ {
    "token" : "весел",
    "start_offset" : 0,
    "end_offset" : 7,
    "type" : "<ALPHANUM>",
    "position" : 0
  }, {
    "token" : "истор",
    "start_offset" : 8,
    "end_offset" : 15,
    "type" : "<ALPHANUM>",
    "position" : 1
  }, {
    "token" : "кот",
    "start_offset" : 20,
    "end_offset" : 25,
    "type" : "<ALPHANUM>",
    "position" : 3
  } ]
}

Iyo standard analyzer yakakamura tambo nenzvimbo uye yakashandura zvese kuita yakaderera, muongorori wekuRussia akabvisa mazwi asina kukosha, akaashandura kuita madiki uye akasiya dzinde remashoko.

Ngationei kuti ndeipi Tokenizer, TokenFilters, CharFilters iyo russian analyzer inoshandisa:

{
  "filter": {
    "russian_stop": {
      "type":       "stop",
      "stopwords":  "_russian_"
    },
    "russian_keywords": {
      "type":       "keyword_marker",
      "keywords":   []
    },
    "russian_stemmer": {
      "type":       "stemmer",
      "language":   "russian"
    }
  },
  "analyzer": {
    "russian": {
      "tokenizer":  "standard",
      /* TokenFilters */
      "filter": [
        "lowercase",
        "russian_stop",
        "russian_keywords",
        "russian_stemmer"
      ]
      /* CharFilters отсутствуют */
    }
  }
}

Ngatitsanangurirei analyzer yedu kubva kuRussia, iyo inocheka html tags. Ngatizviti default, nekuti analyzer ine zita iri ichashandiswa nekusingaperi.

{
  "filter": {
    "ru_stop": {
      "type":       "stop",
      "stopwords":  "_russian_"
    },
    "ru_stemmer": {
      "type":       "stemmer",
      "language":   "russian"
    }
  },
  "analyzer": {
    "default": {
      /* добавляем удаление html тегов */
      "char_filter": ["html_strip"],
      "tokenizer":  "standard",
      "filter": [
        "lowercase",
        "ru_stop",
        "ru_stemmer"
      ]
    }
  }
}

Kutanga, ese ma tag eHTML achabviswa kubva kune tambo tambo, ipapo muyero we tokenizer uchaipatsanura kuita zviratidzo, mhedzisiro inozoenda kune yakaderera, mazwi asingakoshi anobviswa, uye akasara zviratidzo zvicharamba zviri hunde reshoko.

Kugadzira index

Pamusoro takatsanangura iyo default analyzer. Ichashanda kune ese tambo minda. Yedu positi ine ruzhinji rwema tag, saka iwo ma tag anozogadziriswawo neanalyzer. Nokuti Isu tiri kutsvaga zvinotumirwa nechaiyo match kune tag, saka isu tinoda kudzima ongororo yenzvimbo yema tag.

Ngatigadzirei index blog2 ine analyzer uye mepu, umo ongororo yendima yema tag yakadzimwa:

curl -XPOST "$ES_URL/blog2" -d'
{
  "settings": {
    "analysis": {
      "filter": {
        "ru_stop": {
          "type": "stop",
          "stopwords": "_russian_"
        },
        "ru_stemmer": {
          "type": "stemmer",
          "language": "russian"
        }
      },
      "analyzer": {
        "default": {
          "char_filter": [
            "html_strip"
          ],
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "ru_stop",
            "ru_stemmer"
          ]
        }
      }
    }
  },
  "mappings": {
    "post": {
      "properties": {
        "content": {
          "type": "string"
        },
        "published_at": {
          "type": "date"
        },
        "tags": {
          "type": "string",
          "index": "not_analyzed"
        },
        "title": {
          "type": "string"
        }
      }
    }
  }
}'

Ngatiwedzerei zvakafanana zvinyorwa zvitatu kune iyi index (blog3). Ini ndichasiya iyi process nekuti... zvakafanana nekuwedzera magwaro kubhurogu index.

Tsvakiridzo yemavara akazara nerutsigiro rwekutaura

Ngatitarisei imwe mhando yekukumbira:

# найдем документы, в которых встречается слово 'истории'
# query -> simple_query_string -> query содержит поисковый запрос
# поле title имеет приоритет 3
# поле tags имеет приоритет 2
# поле content имеет приоритет 1
# приоритет используется при ранжировании результатов
curl -XPOST "$ES_URL/blog2/post/_search?pretty" -d'
{
  "query": {
    "simple_query_string": {
      "query": "истории",
      "fields": [
        "title^3",
        "tags^2",
        "content"
      ]
    }
  }
}'

Nokuti Isu tiri kushandisa analyzer ine Russian stemming, ipapo chikumbiro ichi chinodzosera zvese zvinyorwa, kunyangwe zviine izwi rekuti 'nhoroondo' chete.

Chikumbiro chinogona kunge chine mavara akakosha, semuenzaniso:

""fried eggs" +(eggplant | potato) -frittata"

Kumbira syntax:

+ signifies AND operation
| signifies OR operation
- negates a single token
" wraps a number of tokens to signify a phrase for searching
* at the end of a term signifies a prefix query
( and ) signify precedence
~N after a word signifies edit distance (fuzziness)
~N after a phrase signifies slop amount

# найдем документы без слова 'щенки'
curl -XPOST "$ES_URL/blog2/post/_search?pretty" -d'
{
  "query": {
    "simple_query_string": {
      "query": "-щенки",
      "fields": [
        "title^3",
        "tags^2",
        "content"
      ]
    }
  }
}'

# получим 2 поста про котиков

nezvakanyorwa

• Elastic.co
• Reference
• Guide
• Tsanangudzo yemashoko
• Kuiswa
• Document manipulation
• Index Operations
• Rondedzero yezvikumbiro

PS

Kana iwe uchifarira zvinyorwa-zvidzidzo zvakadaro, uine mazano ezvinyorwa zvitsva kana uine mazano ekushandira pamwe, ini ndichafara kugamuchira meseji mune yangu meseji kana netsamba m.kuzmin+habr@darkleaf.ru.

Source: www.habr.com