Welcome to the new Box Support website. Check out all the details here on what’s changed.

Uploading large files with box SDK

New post

Comments

2 comments

  • jcleblanc

    Hi ,

     

    What are the problems / errors that you're encountering? We also have these guides available that walk you through the entire process that can help: https://developer.box.com/en/guides/uploads/chunked/

     

    - Jon

    0
    Comment actions Permalink
  • Joseph Nones-MIGRATED

    Hi Box Team,

    I have the same issue at hand.

    I need to upload 5gb size file size to box folder using python.

    What i have done so far:

    (1) First i asked our company box admin to grant me the access tokens to do box transactions.

    And using Box.V2.Core i was able to upload small size files. Which i suppose that tokens defined and on my json file allows me to do the transaction.

    (2) Following the instructions on this link, https://developer.box.com/en/guides/uploads/chunked/..

    I tried to do step by step, copying and pasting the sample code in python to do the 

    Create session, Upload Part and Commit Upload routines.

    I can see that it was able to create the required session, but i got stuck on the uploading part and never to have reached committing the the upload.

    What i wanted to accomplish is to upload such big file and return back to the user the status of the uploading..

     

    Here' the python code :

    ***********************************************************
    from boxsdk import JWTAuth, Client
    from boxsdk.exception import BoxException
    import hashlib


    config = JWTAuth.from_settings_file('132143_mygh48r3_config.json')
    client = Client(config)

    #creating upload session of the file to be uploaded
    file_size = 4596246517
    file_name = 'TestZippedFile.zip'
    upload_session = client.folder('130904095940').create_upload_session(file_size, file_name)
    print('Created upload session {0} with chunk size of {1} bytes'.format(upload_session.id, upload_session.part_size))
    #
    ##upload part
    upload_session_part = client.upload_session(upload_session.id)
    offset = upload_session.part_size * 200
    total_size = 4596246517
    part_bytes = b'abcdefgh'
    part = upload_session.upload_part_bytes(part_bytes, offset, total_size)
    print('Successfully uploaded part ID {0}'.format(part['part_id']))


    #commit upload
    sha1 = hashlib.sha1()
    # sha1 should have been updated with all the bytes of the file

    file_atributes = {
    'description': 'A file uploaded via Chunked Upload',
    }

    upload_session = client.upload_session(upload_session.id)
    uploaded_file = upload_session.commit(sha1.digest(), file_atributes=file_atributes)

    parts = client.upload_session(upload_session.id).get_parts()
    for part in parts:
    print('Part {0} at offset {1} has already been uploaded'.format(part['part_id'], part['offset']))

    print('Successfully uploaded file {0} with description {1}'.format(uploaded_file.id, uploaded_file.description))

    ***********************************************************

    Here's the line by line execution on my console..

    ***********************************************************

    from boxsdk import JWTAuth, Client

    from boxsdk.exception import BoxException

    import hashlib

    config = JWTAuth.from_settings_file('132143_mygh48r3_config.json')

    client = Client(config)

    #creating upload session of the file to be uploaded

    file_size = 4596246517

    file_name = 'TestZippedFile.zip'

    upload_session = client.folder('130904095940').create_upload_session(file_size, file_name)

    Created upload session DB7F6F814BEB57DEEE08FA9D7FCDF76B with chunk size of 33554432 bytes

    upload_session_part = client.upload_session(upload_session.id)

    offset = upload_session.part_size * 200

    total_size = 4596246517

     

     

     

    part_bytes = b'abcdefgh'

    part = upload_session.upload_part_bytes(part_bytes, offset, total_size)


    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:35 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

     

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:37 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    ***********************************************************

     

     

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:40 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''


    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:44 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:50 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:47:01 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    Traceback (most recent call last):

    File "<ipython-input-17-9c0041bc7fb6>", line 1, in <module>
    part = upload_session.upload_part_bytes(part_bytes, offset, total_size)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\object\upload_session.py", line 98, in upload_part_bytes
    data=part_bytes,

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 122, in put
    return self.request('PUT', url, **kwargs)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 158, in request
    response = self._prepare_and_send_request(method, url, **kwargs)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 406, in _prepare_and_send_request
    self._raise_on_unsuccessful_request(network_response, request)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 329, in _raise_on_unsuccessful_request
    network_response=network_response

    BoxAPIException: Message: None
    Status: 500
    Code: None
    Request ID: None
    Headers: {'Date': 'Sun, 21 Feb 2021 10:47:01 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    URL: https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B
    Method: PUT
    Context Info: None


    uploaded_file = upload_session.commit(sha1.digest(), file_atributes=file_atributes)ERROR: execution aborted

     

    Thanks in advance Box Team!

     

     

     

     

     

     

     

     

     

     

    0
    Comment actions Permalink

Please sign in to leave a comment.