• :Apache FTPClient操作“卡死”问题的分析和解决


        最近在和一个第三方的合作中不得已需要使用FTP文件接口。由于FTP Server由对方提供,而且双方背后各自的网络环境都很不单纯等等原因,造成测试环境无法模拟实际情况。测试环境中程序一切正常,但是在部署到生产环境之后发现FTP操作不规律性出现“卡死”现象:程序捕获不到任何异常一直卡着,导致轮巡无法正常工作(由于担心在轮巡时间间隔内处理不能完成,我没有采用类似quartz或者crontab的定时任务,而是采用while-true然后sleep的方式)。

        为了解决这个问题,我首先考虑的是对于FTPClient的使用上没有设置超时时间,于是设置了ConnectTimeout、DataTimeout、DefaultTimeout后在生产环境上继续观察,但是问题依旧没有解决。后来我有些怀疑FTPClient api本身是不是有什么问题,想实在不行自己实现一个超时机制吧,不过还是不甘心,还是想从FTPClient api本身去解决问题。又经过一翻研究之后发现:需要使用被动模式,以下摘抄别人的一段简单描述:
    在项目中使用commons-net-3.0.1.jar实现FTP文件的下载,在windows xp上运行正常,但是放到linux上,却出现问题,程序运行到 FTPClient.listFiles()或者FTPClient.retrieveFile()方法时,就停止在那里,什么反应都没有,出现假死状态。google一把,发现很多人也出现了此类问题,最终在一个帖子里找到了解决办法。在调用这两个方法之前,调用FTPClient.enterLocalPassiveMode();这个方法的意思就是每次数据连接之前,ftp client告诉ftp server开通一个端口来传输数据。为什么要这样做呢,因为ftp server可能每次开启不同的端口来传输数据,但是在linux上,由于安全限制,可能某些端口没有开启,所以就出现阻塞。OK,问题解决。

        于是我回滚了之前的修改,改为被动模式(关于FTP主动/被动模式的解释,这里我不多说了,关注的朋友可以自己查阅)。但是问题依旧。于是能想到的就是最有的绝招:实在不行自己实现一个超时机制吧。经过一翻研究最简单的方式就是使用:Future解决:

    复制代码
     1 public static void main(String[] args) throws InterruptedException, ExecutionException {
     2         final ExecutorService exec = Executors.newFixedThreadPool(1);
     3 
     4         Callable<String> call = new Callable<String>() {
     5             public String call() throws Exception {
     6                 Thread.sleep(1000 * 5);
     7                 return "线程执行完成.";
     8             }
     9         };
    10 
    11         try {
    12             Future<String> future = exec.submit(call);
    13             String obj = future.get(4 * 1000, TimeUnit.MILLISECONDS); // 任务处理超时时间设置
    14             System.out.println("任务成功返回:" + obj);
    15         } catch (TimeoutException ex) {
    16             System.out.println("处理超时啦....");
    17             ex.printStackTrace();
    18         } catch (Exception e) {
    19             System.out.println("处理失败.");
    20             e.printStackTrace();
    21         }
    22         // 关闭线程池
    23         exec.shutdown();
    24         
    25         System.out.println("完毕");
    26     }
    复制代码

    当然了还有很多其他方式:
    http://tech.sina.com.cn/s/2008-07-04/1051720260.shtml
    http://itindex.net/blog/2010/08/11/1281486125717.html
    http://darkmasky.iteye.com/blog/1115047
    http://www.cnblogs.com/wasp520/archive/2012/07/06/2580101.html
    http://coolxing.iteye.com/blog/1476289
    http://www.cnblogs.com/chenying99/archive/2012/10/24/2737924.html

        虽然找到了终极的“必杀技”,但是此时我还是不甘心,还是想从FTPClient api本身去解决问题,但此时看来也别无它他法。只能试试:即设置被动模式又设置超时时间。经过实际测试,发现问题得以解决。下面把我的FTP工具类贴给大家分享,希望能帮到遇到同样问题的人。

    复制代码
      1 import org.apache.commons.net.ftp.FTP;
      2 import org.apache.commons.net.ftp.FTPClient;
      3 import org.apache.commons.net.ftp.FTPFile;
      4 import org.apache.commons.net.ftp.FTPReply;
      5 
      6 import java.io.BufferedInputStream;
      7 import java.io.BufferedOutputStream;
      8 import java.io.File;
      9 import java.io.FileInputStream;
     10 import java.io.FileNotFoundException;
     11 import java.io.FileOutputStream;
     12 import java.io.IOException;
     13 import java.io.InputStream;
     14 import java.io.OutputStream;
     15 import java.net.UnknownHostException;
     16 import java.util.ArrayList;
     17 import java.util.List;
     18 
     19 public class FtpUtil {
     20     public static final String ANONYMOUS_LOGIN = "anonymous";
     21     private FTPClient ftp;
     22     private boolean is_connected;
     23 
     24     public FtpUtil() {
     25         ftp = new FTPClient();
     26         is_connected = false;
     27     }
     28     
     29     public FtpUtil(int defaultTimeoutSecond, int connectTimeoutSecond, int dataTimeoutSecond){
     30         ftp = new FTPClient();
     31         is_connected = false;
     32         
     33         ftp.setDefaultTimeout(defaultTimeoutSecond * 1000);
     34         ftp.setConnectTimeout(connectTimeoutSecond * 1000);
     35         ftp.setDataTimeout(dataTimeoutSecond * 1000);
     36     }
     37 
     38     /**
     39      * Connects to FTP server.
     40      * 
     41      * @param host
     42      *            FTP server address or name
     43      * @param port
     44      *            FTP server port
     45      * @param user
     46      *            user name
     47      * @param password
     48      *            user password
     49      * @param isTextMode
     50      *            text / binary mode switch
     51      * @throws IOException
     52      *             on I/O errors
     53      */
     54     public void connect(String host, int port, String user, String password, boolean isTextMode) throws IOException {
     55         // Connect to server.
     56         try {
     57             ftp.connect(host, port);
     58         } catch (UnknownHostException ex) {
     59             throw new IOException("Can't find FTP server '" + host + "'");
     60         }
     61 
     62         // Check rsponse after connection attempt.
     63         int reply = ftp.getReplyCode();
     64         if (!FTPReply.isPositiveCompletion(reply)) {
     65             disconnect();
     66             throw new IOException("Can't connect to server '" + host + "'");
     67         }
     68 
     69         if (user == "") {
     70             user = ANONYMOUS_LOGIN;
     71         }
     72 
     73         // Login.
     74         if (!ftp.login(user, password)) {
     75             is_connected = false;
     76             disconnect();
     77             throw new IOException("Can't login to server '" + host + "'");
     78         } else {
     79             is_connected = true;
     80         }
     81 
     82         // Set data transfer mode.
     83         if (isTextMode) {
     84             ftp.setFileType(FTP.ASCII_FILE_TYPE);
     85         } else {
     86             ftp.setFileType(FTP.BINARY_FILE_TYPE);
     87         }
     88     }
     89 
     90     /**
     91      * Uploads the file to the FTP server.
     92      * 
     93      * @param ftpFileName
     94      *            server file name (with absolute path)
     95      * @param localFile
     96      *            local file to upload
     97      * @throws IOException
     98      *             on I/O errors
     99      */
    100     public void upload(String ftpFileName, File localFile) throws IOException {
    101         // File check.
    102         if (!localFile.exists()) {
    103             throw new IOException("Can't upload '" + localFile.getAbsolutePath() + "'. This file doesn't exist.");
    104         }
    105 
    106         // Upload.
    107         InputStream in = null;
    108         try {
    109 
    110             // Use passive mode to pass firewalls.
    111             ftp.enterLocalPassiveMode();
    112 
    113             in = new BufferedInputStream(new FileInputStream(localFile));
    114             if (!ftp.storeFile(ftpFileName, in)) {
    115                 throw new IOException("Can't upload file '" + ftpFileName + "' to FTP server. Check FTP permissions and path.");
    116             }
    117 
    118         } finally {
    119             try {
    120                 in.close();
    121             } catch (IOException ex) {
    122             }
    123         }
    124     }
    125 
    126     /**
    127      * Downloads the file from the FTP server.
    128      * 
    129      * @param ftpFileName
    130      *            server file name (with absolute path)
    131      * @param localFile
    132      *            local file to download into
    133      * @throws IOException
    134      *             on I/O errors
    135      */
    136     public void download(String ftpFileName, File localFile) throws IOException {
    137         // Download.
    138         OutputStream out = null;
    139         try {
    140             // Use passive mode to pass firewalls.
    141             ftp.enterLocalPassiveMode();
    142 
    143             // Get file info.
    144             FTPFile[] fileInfoArray = ftp.listFiles(ftpFileName);
    145             if (fileInfoArray == null) {
    146                 throw new FileNotFoundException("File " + ftpFileName + " was not found on FTP server.");
    147             }
    148 
    149             // Check file size.
    150             FTPFile fileInfo = fileInfoArray[0];
    151             long size = fileInfo.getSize();
    152             if (size > Integer.MAX_VALUE) {
    153                 throw new IOException("File " + ftpFileName + " is too large.");
    154             }
    155 
    156             // Download file.
    157             out = new BufferedOutputStream(new FileOutputStream(localFile));
    158             if (!ftp.retrieveFile(ftpFileName, out)) {
    159                 throw new IOException("Error loading file " + ftpFileName + " from FTP server. Check FTP permissions and path.");
    160             }
    161 
    162             out.flush();
    163         } finally {
    164             if (out != null) {
    165                 try {
    166                     out.close();
    167                 } catch (IOException ex) {
    168                 }
    169             }
    170         }
    171     }
    172 
    173     /**
    174      * Removes the file from the FTP server.
    175      * 
    176      * @param ftpFileName
    177      *            server file name (with absolute path)
    178      * @throws IOException
    179      *             on I/O errors
    180      */
    181     public void remove(String ftpFileName) throws IOException {
    182         if (!ftp.deleteFile(ftpFileName)) {
    183             throw new IOException("Can't remove file '" + ftpFileName + "' from FTP server.");
    184         }
    185     }
    186 
    187     /**
    188      * Lists the files in the given FTP directory.
    189      * 
    190      * @param filePath
    191      *            absolute path on the server
    192      * @return files relative names list
    193      * @throws IOException
    194      *             on I/O errors
    195      */
    196     public List<String> list(String filePath) throws IOException {
    197         List<String> fileList = new ArrayList<String>();
    198         
    199         // Use passive mode to pass firewalls.
    200         ftp.enterLocalPassiveMode();
    201         
    202         FTPFile[] ftpFiles = ftp.listFiles(filePath);
    203         int size = (ftpFiles == null) ? 0 : ftpFiles.length;
    204         for (int i = 0; i < size; i++) {
    205             FTPFile ftpFile = ftpFiles[i];
    206             if (ftpFile.isFile()) {
    207                 fileList.add(ftpFile.getName());
    208             }
    209         }
    210         
    211         return fileList;
    212     }
    213 
    214     /**
    215      * Sends an FTP Server site specific command
    216      * 
    217      * @param args
    218      *            site command arguments
    219      * @throws IOException
    220      *             on I/O errors
    221      */
    222     public void sendSiteCommand(String args) throws IOException {
    223         if (ftp.isConnected()) {
    224             try {
    225                 ftp.sendSiteCommand(args);
    226             } catch (IOException ex) {
    227             }
    228         }
    229     }
    230 
    231     /**
    232      * Disconnects from the FTP server
    233      * 
    234      * @throws IOException
    235      *             on I/O errors
    236      */
    237     public void disconnect() throws IOException {
    238 
    239         if (ftp.isConnected()) {
    240             try {
    241                 ftp.logout();
    242                 ftp.disconnect();
    243                 is_connected = false;
    244             } catch (IOException ex) {
    245             }
    246         }
    247     }
    248 
    249     /**
    250      * Makes the full name of the file on the FTP server by joining its path and
    251      * the local file name.
    252      * 
    253      * @param ftpPath
    254      *            file path on the server
    255      * @param localFile
    256      *            local file
    257      * @return full name of the file on the FTP server
    258      */
    259     public String makeFTPFileName(String ftpPath, File localFile) {
    260         if (ftpPath == "") {
    261             return localFile.getName();
    262         } else {
    263             String path = ftpPath.trim();
    264             if (path.charAt(path.length() - 1) != '/') {
    265                 path = path + "/";
    266             }
    267 
    268             return path + localFile.getName();
    269         }
    270     }
    271 
    272     /**
    273      * Test coonection to ftp server
    274      * 
    275      * @return true, if connected
    276      */
    277     public boolean isConnected() {
    278         return is_connected;
    279     }
    280 
    281     /**
    282      * Get current directory on ftp server
    283      * 
    284      * @return current directory
    285      */
    286     public String getWorkingDirectory() {
    287         if (!is_connected) {
    288             return "";
    289         }
    290 
    291         try {
    292             return ftp.printWorkingDirectory();
    293         } catch (IOException e) {
    294         }
    295 
    296         return "";
    297     }
    298 
    299     /**
    300      * Set working directory on ftp server
    301      * 
    302      * @param dir
    303      *            new working directory
    304      * @return true, if working directory changed
    305      */
    306     public boolean setWorkingDirectory(String dir) {
    307         if (!is_connected) {
    308             return false;
    309         }
    310 
    311         try {
    312             return ftp.changeWorkingDirectory(dir);
    313         } catch (IOException e) {
    314         }
    315 
    316         return false;
    317     }
    318 
    319     /**
    320      * Change working directory on ftp server to parent directory
    321      * 
    322      * @return true, if working directory changed
    323      */
    324     public boolean setParentDirectory() {
    325         if (!is_connected) {
    326             return false;
    327         }
    328 
    329         try {
    330             return ftp.changeToParentDirectory();
    331         } catch (IOException e) {
    332         }
    333 
    334         return false;
    335     }
    336 
    337     /**
    338      * Get parent directory name on ftp server
    339      * 
    340      * @return parent directory
    341      */
    342     public String getParentDirectory() {
    343         if (!is_connected) {
    344             return "";
    345         }
    346 
    347         String w = getWorkingDirectory();
    348         setParentDirectory();
    349         String p = getWorkingDirectory();
    350         setWorkingDirectory(w);
    351 
    352         return p;
    353     }
    354 
    355     /**
    356      * Get directory contents on ftp server
    357      * 
    358      * @param filePath
    359      *            directory
    360      * @return list of FTPFileInfo structures
    361      * @throws IOException
    362      */
    363     public List<FfpFileInfo> listFiles(String filePath) throws IOException {
    364         List<FfpFileInfo> fileList = new ArrayList<FfpFileInfo>();
    365         
    366         // Use passive mode to pass firewalls.
    367         ftp.enterLocalPassiveMode();
    368         FTPFile[] ftpFiles = ftp.listFiles(filePath);
    369         int size = (ftpFiles == null) ? 0 : ftpFiles.length;
    370         for (int i = 0; i < size; i++) {
    371             FTPFile ftpFile = ftpFiles[i];
    372             FfpFileInfo fi = new FfpFileInfo();
    373             fi.setName(ftpFile.getName());
    374             fi.setSize(ftpFile.getSize());
    375             fi.setTimestamp(ftpFile.getTimestamp());
    376             fi.setType(ftpFile.isDirectory());
    377             fileList.add(fi);
    378         }
    379 
    380         return fileList;
    381     }
    382 
    383     /**
    384      * Get file from ftp server into given output stream
    385      * 
    386      * @param ftpFileName
    387      *            file name on ftp server
    388      * @param out
    389      *            OutputStream
    390      * @throws IOException
    391      */
    392     public void getFile(String ftpFileName, OutputStream out) throws IOException {
    393         try {
    394             // Use passive mode to pass firewalls.
    395             ftp.enterLocalPassiveMode();
    396             
    397             // Get file info.
    398             FTPFile[] fileInfoArray = ftp.listFiles(ftpFileName);
    399             if (fileInfoArray == null) {
    400                 throw new FileNotFoundException("File '" + ftpFileName + "' was not found on FTP server.");
    401             }
    402 
    403             // Check file size.
    404             FTPFile fileInfo = fileInfoArray[0];
    405             long size = fileInfo.getSize();
    406             if (size > Integer.MAX_VALUE) {
    407                 throw new IOException("File '" + ftpFileName + "' is too large.");
    408             }
    409 
    410             // Download file.
    411             if (!ftp.retrieveFile(ftpFileName, out)) {
    412                 throw new IOException("Error loading file '" + ftpFileName + "' from FTP server. Check FTP permissions and path.");
    413             }
    414 
    415             out.flush();
    416 
    417         } finally {
    418             if (out != null) {
    419                 try {
    420                     out.close();
    421                 } catch (IOException ex) {
    422                 }
    423             }
    424         }
    425     }
    426 
    427     /**
    428      * Put file on ftp server from given input stream
    429      * 
    430      * @param ftpFileName
    431      *            file name on ftp server
    432      * @param in
    433      *            InputStream
    434      * @throws IOException
    435      */
    436     public void putFile(String ftpFileName, InputStream in) throws IOException {
    437         try {
    438             // Use passive mode to pass firewalls.
    439             ftp.enterLocalPassiveMode();
    440             
    441             if (!ftp.storeFile(ftpFileName, in)) {
    442                 throw new IOException("Can't upload file '" + ftpFileName + "' to FTP server. Check FTP permissions and path.");
    443             }
    444         } finally {
    445             try {
    446                 in.close();
    447             } catch (IOException ex) {
    448             }
    449         }
    450     }
    451 }
    复制代码
  • 相关阅读:
    poj 2186(强连通分量)
    zoj 3602
    STL string常用函数
    poj 2503 (map)
    poj 1161 walls
    poj 1164 dfs 位运算
    搭建Hadoop2.0(一)系统环境基本配置
    一步一个脚印——开启博客
    Javascript动态执行问题浅析
    input输入框的各种样式
  • 原文地址:https://www.cnblogs.com/hzcya1995/p/13318174.html
Copyright © 2020-2023  润新知