You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, Im getting this error since mid Mar when before it was working well.
Environment details
OS type and version: deploy in the google cloud functions
Python version: 3.11
pip version: -
google-cloud-storage 2.16.0
Code example
# example
`importloggingimportosimportrandomimportrequestsimportstringfrombs4importBeautifulSoupfromgoogle.cloudimportstoragedefhtml_to_attachment(request_json):
storage_client=storage.Client()
bucket_name=os.environ.get("CLOUD_STORAGE_BUCKET")
bucket=storage_client.bucket(bucket_name)
blob_name=generate_random_string(12) +'.txt'blob=bucket.blob(blob_name)
text_content=html_to_text(request_json['html'])
logging.warning("html to attachment text content is: "+text_content)
blob.upload_from_string(text_content, content_type='text/text')
blob_url=f'https://storage.cloud.google.com/{bucket_name}/{blob_name}'request_json['attachments'] = [{'url': blob_url}]
returnblob_urldefbeyond_to_attachment(request_json):
"""This function is Beyond specific."""storage_client=storage.Client()
bucket_name=os.environ.get("CLOUD_STORAGE_BUCKET")
bucket=storage_client.bucket(bucket_name)
blob_name=generate_random_string(12) +'.txt'blob=bucket.blob(blob_name)
# Extract all the <a> tag textssoup=BeautifulSoup(request_json['html'], 'html.parser')
a_tags=soup.find_all('a')
a_tag_texts= [a_tag.get_text(strip=True) fora_tagina_tags]
# Find the order URL then download the ordera_tag_texts= [textfortextina_tag_textsiftextandtext.startswith('http')]
url=a_tag_texts[0]
response=requests.get(url)
html_content=response.texttext_content=html_to_text(html_content)
blob.upload_from_string(text_content, content_type='text/text')
blob_url=f'https://storage.cloud.google.com/{bucket_name}/{blob_name}'request_json['attachments'] = [{'url': blob_url}]
returnblob_urldefhtml_to_text(html_content):
# Parse the HTML contentsoup=BeautifulSoup(html_content, 'html.parser')
# Process list itemsforliinsoup.find_all('li'):
li.string=f'* {li.string}'li.insert_after('\n')
# Process tablesforrowinsoup.find_all('table'):
row.insert_before('{====\n')
row.insert_after('====}\n')
forrowinsoup.find_all('tr'):
row.insert_before('[--\n')
row.insert_after('--]\n')
forcellinsoup.find_all(['td', 'th']):
cell.insert_after('\n')
processed_text=soup.get_text(separator='\n', strip=True)
returnprocessed_textdefgenerate_random_string(length):
"""Generates a random string of the specified length. Args: length: The desired length of the random string. Returns: A random string consisting of letters and digits. """characters=string.ascii_letters+string.digitsrandom_string=''.join(random.choice(characters) for_inrange(length))
returnrandom_string`
Stack trace
`requests.exceptions.SSLError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries exceeded with url: /upload/storage/v1/b/peblla-order-test/o?uploadType=multipart&ifGenerationMatch=0 (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)')))`
It worked when I used upload_from_string in the html_to_attachment function to upload the file, but when I used beyond_to_attachment to upload the file, it had problems uploading strings, which seemed to be an ssl related error
Thanks!
The text was updated successfully, but these errors were encountered:
Hi @sky-silence . I (and several others) are having similar problems. Can you please put together a minimal reproducing script (your example file is incomplete). It would be very helpful for many fellow storage users! Thanks
Thanks for reporting. We're looking into this, and it seems to be caused by issues in the underlying cpython and urllib3 packages. The urllib3 open issue is not resolved yet, so I'd suggest trying a few workarounds in the meanwhile
Change to use python 3.9 (cpython issue does not affect python versions < 3.10)
Ensure that you have retries enabled for the uploads, either (1) apply if_generation_match preconditions with the upload method, as demonstrated here, or (2) modify to always retry by using DEFAULT_RETRY, as shown below
from google.cloud.storage.retry import DEFAULT_RETRY
upload_from_string(retry=DEFAULT_RETRY)
[cpython]
The cpython bug affects python 3.10+. Errors such as broken pipe error and connection reset by peer that occur when the connection is interrupted during ssl communication have all been changed to SSLEOFError errors.
Bug is fixed in latest python version, but not backported to python 3.10 and 3.11. Features developed in Python 3.9 or under do not need action.
[urllib3]
Open issue not yet resolved. Main impact being that the connection errors are being misinterpreted as SSLEOFError, and the urllib3 library raises theSSLEOFError as a result. Whereas urllib3 gracefully handles broken pipe/connection rest by peer errors.
urllib3.exceptions.SSLError are considered retryable errors for upload operations in the python storage client as long as retry is enabled. However, without further resolution in the underlying urllib3 library, it's possible the SSL errors will exhausts retries if they are exacerbated by overall network instability.
The team still has no success in reproducing this. Being as there is currently a workaround I am going to close this issue. If you feel that this did not resolve your issue please feel free to reopen.
Uh oh!
There was an error while loading. Please reload this page.
Hi, Im getting this error since mid Mar when before it was working well.
Environment details
Code example
Stack trace
It worked when I used upload_from_string in the html_to_attachment function to upload the file, but when I used beyond_to_attachment to upload the file, it had problems uploading strings, which seemed to be an ssl related error
Thanks!
The text was updated successfully, but these errors were encountered: