Rebuild Index always fails unless ClearIndexOnRebuild is set to true

[kennethmyhra]

Describe the bug
Rebuild Index always fails unless ClearIndexOnRebuild is set to true.

To Reproduce
Steps to reproduce the behavior:

1. Make sure that IndexServiceSettings.ClearIndexOnRebuild is set to false
2. Go to https://localhost:5001/admin1. Click on button Reindex
3. The log window will contain messages: Failed to reindex entry VisionPrescription/33123/_history/1
4. Spark logs will contain the corresponding exception:

fail: Spark.Engine.Service.FhirServiceExtensions.IndexRebuildService[0]
      Failed to reindex entry VisionPrescription/33123/_history/1
      MongoDB.Driver.MongoCommandException: Command findAndModify failed: Plan executor error during findAndModify :: caused by :: E11000 duplicate key error collection: spark.searchindex index: internal_id_1 dup key: { internal_id: "VisionPrescription/33123" }.
         at MongoDB.Driver.Core.Operations.RetryableWriteOperationExecutor.ExecuteAsync[TResult](OperationContext operationContext, IRetryableWriteOperation`1 operation, RetryableWriteContext context)
         at MongoDB.Driver.Core.Operations.RetryableWriteOperationExecutor.ExecuteAsync[TResult](OperationContext operationContext, IRetryableWriteOperation`1 operation, IWriteBinding binding, Boolean retryRequested, Int32 maxAdaptiveRetries, Boolean enableOverloadRetargeting, IMayUseSecondaryCriteria mayUseSecondary)
         at MongoDB.Driver.OperationExecutor.ExecuteWriteOperationAsync[TResult](OperationContext operationContext, IClientSessionHandle session, IWriteOperation`1 operation, Boolean allowChannelPinning)
         at MongoDB.Driver.MongoCollectionImpl`1.ExecuteWriteOperationAsync[TResult](IClientSessionHandle session, IWriteOperation`1 operation, Nullable`1 timeout, CancellationToken cancellationToken)
         at MongoDB.Driver.MongoCollectionImpl`1.FindOneAndReplaceAsync[TProjection](FilterDefinition`1 filter, TDocument replacement, FindOneAndReplaceOptions`2 options, CancellationToken cancellationToken)
         at Spark.Store.MongoDB.Search.Common.MongoIndexStore.SaveAsync(BsonDocument document) in /home/kennethmyhra/src/spark/Libraries/Spark.Store.MongoDB/Search/Common/MongoIndexStore.cs:line 59
         at Spark.Store.MongoDB.Search.Common.MongoIndexStore.SaveAsync(IndexValue indexValue) in /home/kennethmyhra/src/spark/Libraries/Spark.Store.MongoDB/Search/Common/MongoIndexStore.cs:line 38
         at Spark.Engine.Service.FhirServiceExtensions.IndexService.IndexResourceAsync(Resource resource, IKey key) in /home/kennethmyhra/src/spark/Libraries/Spark.Engine/Service/FhirServiceExtensions/IndexService.cs:line 64
         at Spark.Engine.Service.FhirServiceExtensions.IndexService.ProcessAsync(Entry entry) in /home/kennethmyhra/src/spark/Libraries/Spark.Engine/Service/FhirServiceExtensions/IndexService.cs:line 48
         at Spark.Engine.Service.FhirServiceExtensions.IndexRebuildService.<>c__DisplayClass9_0.<<RebuildIndexAsync>b__0>d.MoveNext() in /home/kennethmyhra/src/spark/Libraries/Spark.Engine/Service/FhirServiceExtensions/IndexRebuildService.cs:line 99

Expected behavior
Should be able to reindex without failures.

Spark version

• Version: 3.1.0

Operating system + Database

• OS: Linux
• Database: MongoDB