admin管理员组

文章数量:1026989

I am doing one bigger migration using enrich and ingest pipeline. I am getting more slowness when doing multiple reindex simultaneously using the ingest pipeline. So, how many reindex can run simultaneously?. So, Please help me.

I am doing one bigger migration using enrich and ingest pipeline. I am getting more slowness when doing multiple reindex simultaneously using the ingest pipeline. So, how many reindex can run simultaneously?. So, Please help me.

  • elasticsearch
  • nest
  • elasticsearch-8
Share Improve this question asked Nov 16, 2024 at 11:00 bala nbala n 391 silver badge12 bronze badges
Add a comment  | 

1 Answer 1

Reset to default 0

There is no straight answer to your question, but I'll try to provide my thoughts on the matter.

In a nutshell, I would say, that you can put as many parallel reindex jobs as your cluster can handle or as many as the consequent performance hindrance is acceptable to you.

The cluster in general has a set number of threads that handle requests and a queue size that will put in waiting all those that cannot be handled straight away. If you overwhelm the cluster with the parallel reindex requests then documents will be dropped or reindexing will fail.

If you don't mind how long the reindex will take then you can throttle it by using

requests_per_second

This means that you can make reindex as slow and light as needed. The more you throttle the more time it'll be needed to complete. The less you throttle the less performant your cluster will be.

I am doing one bigger migration using enrich and ingest pipeline. I am getting more slowness when doing multiple reindex simultaneously using the ingest pipeline. So, how many reindex can run simultaneously?. So, Please help me.

I am doing one bigger migration using enrich and ingest pipeline. I am getting more slowness when doing multiple reindex simultaneously using the ingest pipeline. So, how many reindex can run simultaneously?. So, Please help me.

  • elasticsearch
  • nest
  • elasticsearch-8
Share Improve this question asked Nov 16, 2024 at 11:00 bala nbala n 391 silver badge12 bronze badges
Add a comment  | 

1 Answer 1

Reset to default 0

There is no straight answer to your question, but I'll try to provide my thoughts on the matter.

In a nutshell, I would say, that you can put as many parallel reindex jobs as your cluster can handle or as many as the consequent performance hindrance is acceptable to you.

The cluster in general has a set number of threads that handle requests and a queue size that will put in waiting all those that cannot be handled straight away. If you overwhelm the cluster with the parallel reindex requests then documents will be dropped or reindexing will fail.

If you don't mind how long the reindex will take then you can throttle it by using

requests_per_second

This means that you can make reindex as slow and light as needed. The more you throttle the more time it'll be needed to complete. The less you throttle the less performant your cluster will be.

本文标签： nestHow many reindex can run simultaneously in elasticsearchStack Overflow

版权声明：本文标题：nest - How many reindex can run simultaneously in elasticsearch - Stack Overflow 内容由热心网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://it.en369.cn/questions/1745659964a2161829.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。