README

By using an Eloquent-like query builder, you can tame the beast without having to worry about Elasticsearch's monstrous syntax.

You can use this package without needing to master Elasticsearch deeply and save your time by not having to memorize the higherarchical nested syntax.

Dependencies

Config file and Elasticsearch migration log

In order to be able to save logs related to migrations and customize your configurations, you need to run command below:

php artisan vendor:publish --tag=elastic

Then migrate your database :

php artisan migrate

Config

After publishing the config file, you will see the following content

return [
    'index_prefix' => env('APP_NAME', 'elasticsearch'),
    'host' => 'http://localhost',
    'port' => 9200,
    'reindex_migration_driver' => "sync", //sync or queue,
    "reindex_migration_queue_name" => 'default',
    'base_migrations_path' => app_path('Elasticsearch/Migrations'),
    'base_models_path' => app_path('Elasticsearch/Models'),
    "username" => env('ELASTICSEARCH_USERNAME', null),
    'password' => env('ELASTICSEARCH_PASSWORD', null),
    'bridges' => [
    'elastic-model-paths' => [
          'app/Elasticsearch/Models'
        ]
    ]
];

ORM

This package has added ORM functionality to make it easier to work with documentation, just like what we see in Laravel.We will get to know more as we go forward.Let's dive into it

Models

to be able to have a more effective relationship with our documents, we need to have a model for each index. Models similar to what we see in Laravel greatly simplify the work of communicating with the database.

In order to create a Model:

php artisan elastic:make-model <model-name>

By default, your models base path is in app/Elasticsearch/Models directory, But you can define your own base path in config/elasticsearch.php file.

All your models must inherit from the BaseElasticsearchModel class. This class is an abstract class that enforce you to implement the getIndex method that returns the index name of model.

public function getIndex():string 

{
    return 'articles';
}

We use the return value of this method to create the index you want in migrations.

If you want to get your index name with the prefix that you defined in config file:

$model->getIndexWithPrefix();

Migrations

As you may know, Elasticsearch uses mappings for the structure of its documents, which may seem a little difficult to create in raw form. In order to simplify this process, we use migrations to make this process easier. After defining the model, you have to create a migration to register your desired fields.All your migrations must inherit from the BaseElasticMigration abstract class.

To Create a new Migration:

php artisan elastic:make-migration <migration-name>

By default, your migrations base path is in app/Elasticsearch/Migrations directory, but you can define your own base path in config/elasticsearch.php file.

<?php

//app/Elasticsearch/Migrations

use Mawebcoder\Elasticsearch\Migration\BaseElasticMigration
use App\Elasticsearch\Models\EArticleModel;


return new class extends BaseElasticMigration {

public function getModel():string 
{

    return EArticleModel::class;
}


 public function schema(BaseElasticMigration $mapper): void
    {
        $mapper->integer('id');
        $mapper->string('name');
        $mapper->boolean('is_active');
        $mapper->text('details');
        $mapper->integer('age');
        $mapper->object('user',function($BaseElasticMigration $mapper){
                                        return $mapper->string('name')
                                        ->object('values',function($mapper){
                                                return $mapper->bigInt('id);
                                        })
                                        });
    }

};

Unfortunately, the package cannot automatically find the path of your migrations. To introduce the path of migrations,put the sample code below in one of your providers:

use Mawebcoder\Elasticsearch\Facade\Elasticsearch;

 public function register(): void
    {
        Elasticsearch::loadMigrationsFrom(__DIR__ . '/../Elastic/Migrations');
        Elasticsearch::loadMigrationsFrom(__DIR__ . '/../App/Migrations');
    }

To see migrations states :

php artisan elastic:migrate-status

To migrate migrations and create your indices mappings :

php artisan elastic:migrate

To reset all migrations(this command just runs down method in all migrations) :

php artisan elstic:migrate --reset

To drop all indices and register them again:

php artisan elastic:migrate --fresh

To rollback migration:

php artisan elastic:migrate-rollback

By default, this command rollbacks the migrations just one step.if you want to determine steps by yourself:

php artisan elastic:migrate-rollback --step=<number>

Field Types

Integer

$this->integer('age');

String(keyword)

$this->string('name');

Object

$this->object('categories',function($mapper){
    return $mapper->string('name','teylor')
-           >integer('parentM_id',22)
            ->object('sequences',function($mapper){
                    return $mapper->bigInt('id');
            })
})

Boolean

$this->boolean('is_active');

SmallInteger(short)

$this->smallInteger('age');

BigInteger(long)

$this->bigInteger('income');

Double

$this->double('price');

Float

$this->float('income');

TinyInt(byte)

$this->tinyInt('value');

Text

$this->text(field:'description',fieldData:true);

DateTime(date)

$this->datetime('created_at');

Edit Indices Mappings

Sometimes you need modify your mappings. To do this you have to add a new migration:

php artisan elastic:make-migration <alter migration name>

<?php

use Mawebcoder\Elasticsearch\Migration\BaseElasticMigration;
use Mawebcoder\Elasticsearch\Models\EArticleModel;
use Mawebcoder\Elasticsearch\Migration\AlterElasticIndexMigrationInterface;

return new class extends BaseElasticMigration implements AlterElasticIndexMigrationInterface {
    public function getModel(): string
    {
        return EArticleModel::class;
    }

    public function schema(BaseElasticMigration $mapper): void
    {
        $mapper->dropField('name');
        $mapper->boolean('new_field');//add this field to 
    }

    public function alterDown(BaseElasticMigration $mapper): void
    {
        $mapper->string('name');
        $mapper->dropField('new_field');
    }
};

As you can see, we implemented AlterElasticIndexMigrationInterface interface in our migration. Then in alterDown method we wrote our rollback scenario. Finally, migrate your migration:

php artisan elastic:migrate

Dynamic Mapping

By default, Elasticsearch detects the type of fields that you have not introduced in mapping and defines its type automatically. The package has disabled it by default. To activate it, do the following in your migration:

protected bool $isDynamicMapping = true;

Query Builder

Just like Laravel, which enabled you to create complex and crude queries by using a series of pre-prepared methods, this package also uses this feature to give you a similar experience.

Store a recorde

$eArticleModel=new EArticleModel();

$eArticleModel->name='mohammad';

$eArticleModel->id=2;

$eArticleModel->is_active=true;

$eArticleModel->user=[
    'name'=>'komeil',
    'id'=>3
];

$eArticleModel->text='your text';

$eArticleModel->age=23;

$eArticleModel->save();

• Note: If you don't pass any field that exists in your mappings,we set that as null by default

Store Several Records (Bulk Insert)

$users = [
    [
        id => 1,
        name => 'Mohsen',
        is_active => true,
        text => 'your text',
        age => 25
    ],
    [
        id => 2,
        name => 'Ali',
        is_active => true,
        text => 'your text',
        age => 20
    ]
];

$result=EUserModel::newQuery()->saveMany($users);

Sometimes, some items may not be saved in the database due to an error. you can check this like below:

if($result->hasError()){
    $notImportedItems=$result->getNotImportedItems();
}

$importedItems=$result->getImportedItems();

Also if you want to rollback transaction if any error happend set the $withTransaction argumet as true:

$result=EUserModel::newQuery()->saveMany(items:$users,withTransaction:true);

this action will remove the imported items from database.

Find record

$eArticleModel=new EArticleModel();

$result=$eArticleModel->find(2);

Remove record

$eArticleModel=new EArticleModel();

$result=$eArticleModel->find(2);

$result?->delete();

Conditions

Equal

$eArticleModel
->where('id',2)
->orWhere('id',2)
->get();

Not Equal

$eArticleModel
->where('id','<>',2)
->orWhere('id','<>',10)
->get();

• Note:Do not use =,<> operators on text fields because we used term in these operators.in text field you need to use like or not like operator instead

Greater Than

$eArticleModel->where('id','>',10)->orWhere('id','>=',20)->get();

Lower Than

$eArticleModel->where('id','<',2)->orWhere('id',null)->first();

Like

$eArticleModel->where('name','like','foo')->orWhere('name','not like','foo2')->get();

whereTerm

Sometimes you want to search for a specific phrase in a text. In this case, you can do the following :

$eArticleModel->whereTerm('name','foo')->orWhereTerm('name','<>','foo2')->get();

whereIn

$eArticleModel->whereIn('id',[1,2])->orWhereIn('age',[23,26])->first();

whereNotIn

$eArticleModel->whereNotIn('id',[1,2])->orWhereNotIn('id',[26,23])->get();

whereBetween

$eArticleModel->whereBetween('id,[1,2])->orWhereBetween('age',[26,27])->first();

whereNotBetween

$eArticleModel->whereNotBetween('id',[1,2])->orWhereNotBetween('id,'26,27])->get();

whereNull

$eArticleModel->whereNull('id')->orWhereNull('name')->first();

whereNotNull

$eArticleModel->whereNotNull('id')->orWhereNotNull()->first();

Chaining

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->orWhere('name,'komeil')
->whereNotIn('id,[1,2,3])
->where('name','like','value')
->whereBetween('id',[10,13])
->whereNotBetween('id',[1,2])
->whereTerm('name','foo')
->get();

Fuzzy Search

Note: fuzzy search just works on the text fields

$eArticleModel=new EArticleModel();

$eArticleModel
->whereFuzzy('name','mawebcoder',fuzziness:3)
->get();

You can change the fuzziness value as you want

Get pure Query

$eArticleModel->where('id','like','foo')->dd();

Update record

$eArticleModel=new EArticleModel();

$result=$eArticleModel->find(2);

$result?->update([
    'name'=>'elastic',
    //...
]);

Bulk update

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->update([
    'name'=>'elastic',
    //...
]);

Bulk delete

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->delete();

Take(limit)

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->take(10)
->get();

Offset

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->take(10)
->offset(5)
->get();

select

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->select('name,'age','id')
->get();

OrderBy

$eArticleModel=new EArticleModel();

$eArticleModel
->where('name','<>','mohammad')
->select('name,'age')
->orderBy('age','desc')
->get();

• Note:Do not use orderBy on text fields because text is not optimized for sorting operation in Elasticsearch.But if you want to force to sort the texts set the fielddata=true:

$this->text(field:"description",fielddata:true) 

By Adding above line you can use texts as sortable and in aggregations,but fielddata uses significant memory while indexing

Get specific field's value

$name=$eArticleModel->name;

Nested Search

First of all we need to define object type in our migration:

"category":{
            "name":"elastic",
            "id":2
          }

$mapper->object('categories',[
            'name'=>self::TYPE_STRING,
            'id'=>self::TYPE_BIGINT
        ]);

$eArticleModel
->where('categories.name')->first();

If you have multi dimension objects like below:

"categories":[
           {
            "name":"elastic",
            "id":2
           },
           {
            "name":"mohammad",
            "id":3
           }


            ]

Define your mappings Like below:

$mapper->object('categories',[
            'name'=>self::TYPE_STRING,
            'id'=>self::TYPE_BIGINT
        ]);

$eArticleModel
->where('categories.name')->first();

Destroy by id

$eArticleModel->destroy([1,2,3]);

Nested Queries

In order to create complex and nested queries, you can use the nesting function of the builder. There is no limit to the nesting of your queries:

$model=Model::newQuery()
->where('name','john')
->where(function($model){
        return $model->where('age',22)
        ->orWhere(function($model){
        return $model->whereBetween('range',[1,10]);
        })
})->orWhere(function($model){
    return $model->where('color','red')
    ->orWhereIn('cars',['bmw','buggati'])
})->get()

Just pay attention that you need to return the queries inside closure otherwise it will be ignored

chunk

for better managing your memory usage you can use the chunk method :

$model=Model::newQuery()
->where('name','mohammad')
->chunk(100,function(Collection $collection){
    //code here
})

Aggregations

By default, all related data also will be return, If you want just aggregations be in your result use take(0) to prevent oveloading data in you request

Count

$eArticleModel
->where('categories.name')
->orWhere('companies.title','like','foo')
->count();

bucket

$eArticleModel
->bucket('languages','languages-count')
->get();

$aggregations=$eArticleModel['aggregations'];

By default, bucket method returns maximum 2147483647 number of the records,if You want to change it:

$eArticleModel
->bucket('languages','languages-count',300) //returns 300 records maximum
->get();

Min

$model->min('year')->get()

Max

$model->max('year')->get()

Avg

$model->avg('year')->get()

Sum

$model->sum('year')->get()

Unique

Sometimes you need to retrieve only unique records based on an criterion:

$model->where('category','elastic')->unique('name');

groupBy

In order to group data based on a criterion

$model->where('category','elastic')->groupby(field:'age',sortField:'category',direction:'desc');

Pagination

$eArticleModel
->where('categories.name')
->orWhere('companies.title','like','foo')
->paginate()

By default paginate methods paginates pages per 15 items,but you can change it:

$eArticleModel
->where('categories.name')
->paginate(20)

The result will be something like this:

[
    "data"=>[...],
    'total_records' => 150,
    'last_page' => 12,
    'current_page' => 9,
    'next_link' => localhost:9200?page=10,
    'prev_link' => localhost:9200?page=8,
]

inRandomOrder

Sometimes you need to get random data from elasticsearch:

$eArticleModel
->where('categories.name')
->inRandomOrder()
->take(10)
->get()

Interact With Documentations

Drop indices by name

Elasticsearch::dropIndexByName($model->getIndexWithPrefix())

Check index Exists

Elasticsearch::hasIndex($model->getIndexWithPrefix())

Get all indices

Elasticsearch::getAllIndexes()

Drop index by model

Elasticsearch::setModel(EArticle::class)->dropModelIndex()

Get all model fields

Elasticsearch::setModel(EArticle::class)->getFields()

Get model mappings

Elasticsearch::setModel(EArticle::class)->getMappings()

Automate Syncing with Primary Database

In NoSQL database systems, merging tables into a single document in Elasticsearch is common. However, when entities are updated in the relational database, keeping Elasticsearch synchronized poses a significant challenge for developers. This often requires writing extensive code to manage dependencies.

To simplify this process, Elasticquent offers a built-in feature called the Bridge, which allows developers to manage dependencies effortlessly.

Example

Consider an entity like an article linked to a category in our relational database.

Articles Table

Categories Table

When storing this data in Elasticsearch, we join these tables.

Elasticsearch Index (Document Structure)

{
  "articles": [
    {
      "id": 1,
      "title": "Sample Article",
      "content": "This is the content of the sample article.",
      "category": {
        "id": 1,
        "title": "Technology",
        "description": "Articles related to technology trends and innovations.",
        "created_at": "2024-04-10T08:00:00Z",
        "updated_at": "2024-04-10T08:00:00Z"
      },
      "created_at": "2024-04-10T08:00:00Z",
      "updated_at": "2024-04-10T08:00:00Z"
    },
    {
      "id": 2,
      "title": "Another Article",
      "content": "This is another sample article.",
      "category": {
        "id": 2,
        "title": "Science",
        "description": "Articles related to scientific discoveries and research.",
        "created_at": "2024-04-10T08:00:00Z",
        "updated_at": "2024-04-10T08:00:00Z"
      },
      "created_at": "2024-04-10T08:00:00Z",
      "updated_at": "2024-04-10T08:00:00Z"
    }
  ]
}

But what if the title of a category is updated in the relational database?

To handle this, we define a bridge in the Elasticsearch model to connect these segments of data.

Defining the Bridge

To create a bridge, implement the SyncInstructionsBridge or CustomSyncBridge interfaces. In this example, we use the SyncInstructionsBridge interface to specify the type of bridge.

<?php

class ArticleCategoryBridge implements SyncInstructionsBridge
{
    public function id(): string
    {
        return 'category.id';
    }

    public static function model(): string
    {
        return Category::class;
    }

    public function updateInstruction(Model $updatedEntity)
    {
        return [
            'category.name' => $updatedEntity->{Category::COLUMN_NAME}
        ];
    }
}

We also have another type of interface for syncing, CustomSyncBridge. This type of syncing is highly custom, allowing you to access the document and update the entity simultaneously.

<?php

class ArticleCategoryBridge implements CustomSyncBridge
{
    public function id(): string
    {
        return 'category.id';
    }

    public static function model(): string
    {
        return Category::class;
    }

    public function updateDocument(BaseElasticsearchModel $document, Model $updatedEntity)
    {
        $document->update(['category.title' => $updatedEntity->name];
        // Or any custom activity that you want to do!
    }

}

Connecting the Bridge to Elasticsearch Model

<?php

class Article extends BaseModel implements ShouldModelSyncByDataBridges
{
    public static function getDataBridges() : string|array
    {
        return CategoryDataBridge::class;
    }
}

What's Next?

The good news is, there's nothing more to do! Simply define the bridge and introduce it into your Elasticsearch model. Then, set a little configuration once, allowing the Elastic package to detect your Elasticsearch models with the autodiscovery feature built into the Elastic package.

In the example above, when a category is updated, this bridge is triggered to update all related items in Elasticsearch based on your bridge definition. 🚀

Auto Discovery of Elastic Models

By default, the package searches the project in the app/Elasticsearch/Models directory to find your Elasticsearch models. You can customize this behavior in your Elasticsearch configuration file.

'bridges' => [
        'elastic-model-paths' => [
            'app/Elasticsearch/Models',
            'infrastructure/*/Elasticsearch/Models', // 👈️ Add your custom path pattern like this (using wildcard)
        ]
    ]

Danger: If your model is not discovered, the bridge may not work successfully. Ensure that your Elasticsearch models are located in the specified directories to avoid issues with model discovery and bridge functionality.

Coming soon

• Histogram
• Define normalizer and tokenizer
• Raw Queries