Level up your Box knowledge with brand new learning paths on Box University. Visit training.box.com to get started

Box Status

Uploading large files with box SDK

Follow
New post
mv891433

Hello everyone,

 

Currently I'm working in a solution to upload large files to Box, here are my requirements:

 

- Language = Python (I have to integrate another solution)

- Create a folder there and upload the files 

- File size: right now is 7.33 GB

 

I'm using JWT as authentication method, but I'm struggling using the method Chunked Uploader, can someone please point me in the right direction?

 

So far, this is what I have

 

rom boxsdk import JWTAuth
from boxsdk import Client

#  Read JSON configuration
auth = JWTAuth.from_settings_file('/FilePath/to/config.json')

# Initialize SDK client
client = Client(auth)
service_account = client.user().get()
print('Service Account user ID is {0}'.format(service_account.id))

# Set upload values
file_path = '/File/Path'
file_name = 'test1.txt'
folder_id = '0'

# Upload the files Chunked
chunked_uploader = client.file(file_name).get_chunked_uploader(file_path)
try:
    uploaded_file = chunked_uploader.start()
except:
    uploaded_file = chunked_uploader.resume()
print('File "{0}" uploaded to Box with file ID {1}'.format(uploaded_file.name, uploaded_file.id))

 

0

Comments

2 comments

Date Votes
  • jcleblanc

    Hi ,

     

    What are the problems / errors that you're encountering? We also have these guides available that walk you through the entire process that can help: https://developer.box.com/en/guides/uploads/chunked/

     

    - Jon

    0
    Comment actions Permalink
  • Joseph Nones-MIGRATED

    Hi Box Team,

    I have the same issue at hand.

    I need to upload 5gb size file size to box folder using python.

    What i have done so far:

    (1) First i asked our company box admin to grant me the access tokens to do box transactions.

    And using Box.V2.Core i was able to upload small size files. Which i suppose that tokens defined and on my json file allows me to do the transaction.

    (2) Following the instructions on this link, https://developer.box.com/en/guides/uploads/chunked/..

    I tried to do step by step, copying and pasting the sample code in python to do the 

    Create session, Upload Part and Commit Upload routines.

    I can see that it was able to create the required session, but i got stuck on the uploading part and never to have reached committing the the upload.

    What i wanted to accomplish is to upload such big file and return back to the user the status of the uploading..

     

    Here' the python code :

    ***********************************************************
    from boxsdk import JWTAuth, Client
    from boxsdk.exception import BoxException
    import hashlib


    config = JWTAuth.from_settings_file('132143_mygh48r3_config.json')
    client = Client(config)

    #creating upload session of the file to be uploaded
    file_size = 4596246517
    file_name = 'TestZippedFile.zip'
    upload_session = client.folder('130904095940').create_upload_session(file_size, file_name)
    print('Created upload session {0} with chunk size of {1} bytes'.format(upload_session.id, upload_session.part_size))
    #
    ##upload part
    upload_session_part = client.upload_session(upload_session.id)
    offset = upload_session.part_size * 200
    total_size = 4596246517
    part_bytes = b'abcdefgh'
    part = upload_session.upload_part_bytes(part_bytes, offset, total_size)
    print('Successfully uploaded part ID {0}'.format(part['part_id']))


    #commit upload
    sha1 = hashlib.sha1()
    # sha1 should have been updated with all the bytes of the file

    file_atributes = {
    'description': 'A file uploaded via Chunked Upload',
    }

    upload_session = client.upload_session(upload_session.id)
    uploaded_file = upload_session.commit(sha1.digest(), file_atributes=file_atributes)

    parts = client.upload_session(upload_session.id).get_parts()
    for part in parts:
    print('Part {0} at offset {1} has already been uploaded'.format(part['part_id'], part['offset']))

    print('Successfully uploaded file {0} with description {1}'.format(uploaded_file.id, uploaded_file.description))

    ***********************************************************

    Here's the line by line execution on my console..

    ***********************************************************

    from boxsdk import JWTAuth, Client

    from boxsdk.exception import BoxException

    import hashlib

    config = JWTAuth.from_settings_file('132143_mygh48r3_config.json')

    client = Client(config)

    #creating upload session of the file to be uploaded

    file_size = 4596246517

    file_name = 'TestZippedFile.zip'

    upload_session = client.folder('130904095940').create_upload_session(file_size, file_name)

    Created upload session DB7F6F814BEB57DEEE08FA9D7FCDF76B with chunk size of 33554432 bytes

    upload_session_part = client.upload_session(upload_session.id)

    offset = upload_session.part_size * 200

    total_size = 4596246517

     

     

     

    part_bytes = b'abcdefgh'

    part = upload_session.upload_part_bytes(part_bytes, offset, total_size)


    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:35 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

     

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:37 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    ***********************************************************

     

     

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:40 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''


    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:44 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:46:50 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    "PUT https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B" 500 0
    {'Date': 'Sun, 21 Feb 2021 10:47:01 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    b''

    Traceback (most recent call last):

    File "<ipython-input-17-9c0041bc7fb6>", line 1, in <module>
    part = upload_session.upload_part_bytes(part_bytes, offset, total_size)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\object\upload_session.py", line 98, in upload_part_bytes
    data=part_bytes,

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 122, in put
    return self.request('PUT', url, **kwargs)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 158, in request
    response = self._prepare_and_send_request(method, url, **kwargs)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 406, in _prepare_and_send_request
    self._raise_on_unsuccessful_request(network_response, request)

    File "C:\Users\1000241831\AppData\Local\Continuum\anaconda3\lib\site-packages\boxsdk\session\session.py", line 329, in _raise_on_unsuccessful_request
    network_response=network_response

    BoxAPIException: Message: None
    Status: 500
    Code: None
    Request ID: None
    Headers: {'Date': 'Sun, 21 Feb 2021 10:47:01 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '0', 'Connection': 'keep-alive', 'Strict-Transport-Security': 'max-age=31536000'}
    URL: https://upload.box.com/api/2.0/files/upload_sessions/DB7F6F814BEB57DEEE08FA9D7FCDF76B
    Method: PUT
    Context Info: None


    uploaded_file = upload_session.commit(sha1.digest(), file_atributes=file_atributes)ERROR: execution aborted

     

    Thanks in advance Box Team!

     

     

     

     

     

     

     

     

     

     

    0
    Comment actions Permalink

Please sign in to leave a comment.