Level up your Box knowledge with brand new learning paths on Box University. Visit training.box.com to get started

Box Status

Getting Folders that are not Empty

Answered
Follow
New post
JC

I have roughly 38,000 folders I am looping through and getting the files in them and pushing them to another system.    The first issue is that the List Items call in the API doesn't seem to have a way to tell it to only pass back folders when it isn't empty. Is that correct?

The second issue is that since the call only gets the folders and files in the one folder ID I am calling, I built a recursive loop to dive into the sub folders and then return the files until there are no more sub folders.

This means that since there is a 1000 limit call per minute then it will take me about 38 minutes to call all the folders and get the contents?  Because I will have to do 1000 folders, pause the application and wait till the minute is up and the continue.  Do I understand this correctly?

Is there a better way to do this?  

Thanks, J

0

Comments

2 comments

Date Votes
  • Jason

    Hi,

     

    That understanding sounds correct. You can theoretically use the "item_collection" or even "size" field on any individual folder to see if that particular folder is empty or not, but the API doesn't offer a way to do something like "give me all non-empty folders underneath this parent folder".

    As for the rate limits, that sounds correct but the logic could be simplified if you're using our SDKs. The SDKs will go as fast as either your code or the rate limits allow, and automatically handle things like retries and backoffs. 

    Finally, there's a slightly different pattern that I sometimes see - rather than a periodic scan and sync, I've seen developers do an an initial scan and then listen for events via the User events stream:

    https://developer.box.com/guides/events/for-user/

    Hope that helps!

    0
    Comment actions Permalink
  • JC

    Thanks Jason for confirming!  I think the approach I am going to take is using a webhook to keep most of the folders and files up to date and then just run a sync once a day late at night to make sure nothing was missed.  I really like how box.com retries multiple times if the web hook doesn't respond which makes it nice if the web service goes down unlike an API like slack that just tries one time and it is done!

    Thanks again for the reply!

    1
    Comment actions Permalink

Please sign in to leave a comment.