Box API download file error
Hi Box,
Got an error while calling though box API to download files from box. And it occasionally failed in the first time job triggered and will be able to retry and download successfully second time. Any thoughts why failed first time?
Here are some details:
550 Box: Service is temporarily unavailable (code=unavailable, request_id=)
LOGS -
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX conn.cwd(path)
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX File "/usr/lib/python3.6/ftplib.py", line 631, in cwd
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX return self.voidcmd(cmd)
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX File "/usr/lib/python3.6/ftplib.py", line 278, in voidcmd
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX return self.voidresp()
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX File "/usr/lib/python3.6/ftplib.py", line 251, in voidresp
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX resp = self.getresp()
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX File "/usr/lib/python3.6/ftplib.py", line 246, in getresp
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX raise error_perm(resp)
[2019-11-13 19:00:38,040] {base_task_runner.py:115} INFO - Job 6247: Subtask DOWNLOAD_FILES_FROM_BOX ftplib.error_perm: 550 Box: Service is temporarily unavailable (code=unavailable,
Airflow Python Function:
def download_files_from_ftp_tls(conn_id, remote_path, file_name, local_path, like_file_name=False, timestamp=False, archive=False, arch_folder='Archive'): """ Download the files from box ftps to local folder and add timestamp to each file on box if already downloaded and archive it. :param remote_path: remote folder :type remote_path: str :param file_name: remote file name such as location.csv :type file_name: str :param local_path: local folder path :type local_path: str :param like_file_name: if file_name is a substring of group files. :type like_file_name: bool :param timestamp: add timestamp to original file name downloaded. :type timestamp: bool :param archive : put the file into archive folder once downloaded. :type archive: bool :param arch_folder: archive folder name, default to Archive :type arch_folder: str """ #clean up remote_path and local path with last / remote_path = remote_path.strip('/') local_path = local_path.strip('/') logging.info("CUST_LOG: download /{}/{} started...".format(remote_path, file_name)) with FTPSHook(ftp_conn_id=conn_id) as ftps: files = ftps.list_directory('/{}/'.format(remote_path)) dlist = [each for each in files if like_file_name and file_name in each] if like_file_name \ else [each for each in files if file_name == each] # for each file in the dlist for each_file in dlist: file_name_timestamp = add_time_stamp(each_file) if timestamp else each_file ori_file = "/{}/{}".format(remote_path, each_file) rnmd_file = "/{}/{}".format(remote_path, file_name_timestamp) arch_file = "/{}/{}/{}".format(remote_path, arch_folder, file_name_timestamp) local_file = "/{}/{}".format(local_path, file_name_timestamp) # download file and archive try: ftps.retrieve_file(ori_file, local_file) except ftplib.error_perm as e: if e.args[0][:3] == '550': logging.info("CUST_LOG: there is no such files uploaded...") else: raise Exception(e) else: # if no error then rename the existing file and archive it if timestamp: ftps.rename(ori_file, rnmd_file) logging.info("CUST_LOG: timestamp files...") if archive: ftps.rename(rnmd_file, arch_file) logging.info("CUST_LOG: archive files...") logging.info("CUST_LOG: Check or download /{}/{} finished...".format(remote_path, file_name))
Also, when I add passive = true into the ftp_tls call then the error changed to : 111 Connection Refused.
-
Hi ,
This could be related to some potential degradation reported, that is being investigated. I would recommend referring to https://status.box.com/ for the most up to date information on this.
Thanks,
Jon
サインインしてコメントを残してください。
コメント
3件のコメント