Skip to content

Failure to upload to storage service #149

Closed
@mocsharp

Description

@mocsharp

Description

The upload service occasionally fails to upload (C-STORE) received data.

<4> 10:35:18 Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService[4000] => File ID=1.2.826.0.1.3680043.2.1125.1.19616861412188316212577695277886020/1.2.826.0.1.3680043.2.1125.1.34918616334750294149839565085991567/1.2.826.0.1.3680043.2.1125.1.68845315069119806578612768053737857, Correlation ID=8c3a300c-aca0-45fd-9907-4b8f77f6f4c1 Failed to upload file 1.2.826.0.1.3680043.2.1125.1.19616861412188316212577695277886020/1.2.826.0.1.3680043.2.1125.1.34918616334750294149839565085991567/1.2.826.0.1.3680043.2.1125.1.68845315069119806578612768053737857; added back to queue for retry. System.NullReferenceException: Object reference not set to an instance of an object.    at Minio.MinioClient.PutObjectAsync(PutObjectArgs args, CancellationToken cancellationToken) in /root/.q/sources/minio-dotnet/Minio/ApiEndpoints/ObjectOperations.cs:line 710    at Monai.Deploy.Storage.MinIO.MinIoStorageService.PutObjectUsingClient(MinioClient client, String bucketName, String objectName, Stream data, Int64 size, String contentType, Dictionary`2 metadata, CancellationToken cancellationToken)    at Monai.Deploy.Storage.MinIO.MinIoStorageService.PutObjectAsync(String bucketName, String objectName, Stream data, Int64 size, String contentType, Dictionary`2 metadata, CancellationToken cancellationToken)    at Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService.<>c__DisplayClass22_0.<<UploadData>b__1>d.MoveNext() in /app/src/InformaticsGateway/Services/Storage/ObjectUploadService.cs:line 226 --- End of stack trace from previous location ---    at Polly.AsyncPolicy.<>c__DisplayClass40_0.<<ImplementationAsync>b__0>d.MoveNext() --- End of stack trace from previous location ---    at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)    at Polly.AsyncPolicy.ExecuteAsync(Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)    at Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService.UploadData(String identifier, StorageObjectMetadata storageObjectMetadata, String source, List`1 workflows, CancellationToken cancellationToken) in /app/src/InformaticsGateway/Services/Storage/ObjectUploadService.cs:line 228    at Monai.Deploy.InformaticsGateway.Services.Storage.ObjectUploadService.ProcessObject(FileStorageMetadata blob) in /app/src/InformaticsGateway/Services/Storage/ObjectUploadService.cs:line 179

Steps to reproduce

Please share a clear and concise description of the problem.

  1. C-STORE a DICOM dataset
  2. Review the logs
    ...

Expected behavior

All files are uploaded to storage service

Actual behavior

One or more files failed to upload

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions