Level up your Box knowledge with brand new learning paths on Box University. Visit training.box.com to get started

Box Status

Clone or download a folder from box to a linux run cluster, an easy to understand method.

Follow
New post
vaksmanz

I am brand new to box and I need to understand, in an easy to understand method or a youtube video (would be great too), how to download (or clone) a folder with a large dataset (400 files and over 1TB total size) to a linux run cluster. 

 

I can not understand the methods that are presented in any of the help modules, they are too broken up with components and require some understanding of how box apps work which I don't. 

 

 

0

Comments

4 comments

Date Votes
  • Nayak

    hi , You would need to use List folder items API first and for each of the items in the folder call Download File and save the binary content accordingly.

     

    Also if this is a one time activity and can be completed in less than 1 hour, you can skip the hassle of authenticating by using a developer token generated directly in the application config.

     

    - Nayak

    0
    Comment actions Permalink
  • vaksmanz

    So I did the following, exactly what it said to do.

     

    I got a token and ran : the Authorization is different of course 

     

    ```

    curl -X GET https://api.box.com/2.0/folders/0/items -H 'Authorization: Bearer tQZMYe9alNd5W6wnK2A1eTBeEldE' -L

    ```

    results were :

     

    ```

    {"total_count":29,"entries":{"type":"file_version","id":"391168840745","sha1":"23ed01b38416827ccb33b9e5ba8af04fab07e75d"},"sequence_id":"0","etag":"0","sha1":"23ed01b38416827ccb33b9e5ba8af04fab07e75d","name":"PCA_His_cohorts_hapmap.pdf"},{"type":"file","id":"369988061574","file_version":{"type":"file_version","id":"391166475174","sha1":"05b86399d3d333473c20f5208c0bc9b113b3b239"},"sequence_id":"0","etag":"0","sha1":"05b86399d3d333473c20f5208c0bc9b113b3b239","name":"Presentation1.pptx"},{"type":"file","id":"369988832927","file_version":{"type":"file_version","id":"391167343727","sha1":"097ee3be8afd32165dd468f7ff67eb6ad8bc930d"},"sequence_id":"0","etag":"0","sha1":"097ee3be8afd32165dd468f7ff67eb6ad8bc930d","name":"Presentation2 [Autosaved].pptx"},{"type":"file","id":"369998202572","file_version":{"type":"file_version","id":"391178298572","sha1":"fe32eb48810fedbd9a9f855218115853cbd30dc4"},"sequence_id":"0","etag":"0","sha1":"fe32eb48810fedbd9a9f855218115853cbd30dc4","name":"ZV_GWAS_paper_11-13-18.pptx"}],"offset":0,"limit":100,"order":[{"by":"type","direction":"ASC"}

    ```

     

    ```curl -X GET https://api.box.com/2.0/folders/0/369989013980/content -H 'Authorization: Bearer tQZMYe9alNd5W6wnK2A1eT' -L```

     

    I got the following message: 

    ```

    {"type":"error","status":404,"code":"not_found","help_url":"http:\/\/developers.box.com\/docs\/#errors","message":"Not Found","request_id":"goakehgdcaswxrau"}

    ```

     

     

     

    0
    Comment actions Permalink
  • Nayak

    Hi , the second API /content works only for files and not on folders. For each folder you need to iterate recursively again if you want to download the same.

    0
    Comment actions Permalink
  • iancrew

    Maybe using rclone (https://rclone.org/box/) or FTP (https://support.box.com/hc/en-us/articles/360043697414-Using-Box-with-FTP-or-FTPS) would be easier than using curl?

    0
    Comment actions Permalink

Please sign in to leave a comment.