• hdfs 路径不支持‘:’


    hdfs 路径不支持‘:’,但Linux支持。

    -put: Pathname /public/platform/control_center/file_center/weiflow/core-user-core-submit-core-dispatch-train_988-1527240620176599/:q from /public/platform/control_center/file_center/weiflow/core-user-core-submit-core-dispatch-train_988-1527240620176599/:q is not a valid DFS filename.

    可以查看此文件:

    /Users/suanec/ksp/yarn/hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java

    org.apache.hadoop.hdfs.DFSUtilClient
      /**
       * Whether the pathname is valid.  Currently prohibits relative paths,
       * names which contain a ":" or "//", or other non-canonical paths.
       */
      public static boolean isValidName(String src) {
        // Path must be absolute.
        if (!src.startsWith(Path.SEPARATOR)) {
          return false;
        }
    
        // Check for ".." "." ":" "/"
        String[] components = StringUtils.split(src, '/');
        for (int i = 0; i < components.length; i++) {
          String element = components[i];
          if (element.equals(".")  ||
              (element.contains(":"))  ||
              (element.contains("/"))) {
            return false;
          }
          // ".." is allowed in path starting with /.reserved/.inodes
          if (element.equals("..")) {
            if (components.length > 4
                && components[1].equals(".reserved")
                && components[2].equals(".inodes")) {
              continue;
            }
            return false;
          }
          // The string may start or end with a /, but not have
          // "//" in the middle.
          if (element.isEmpty() && i != components.length - 1 &&
              i != 0) {
            return false;
          }
        }
        return true;
      }

    hdfs当前方法不支持相对路径,不支持路径中有":"、"/",不支持其他非标准路径格式。

    hdfs的DistributedFileSystem,利用Path的api做了相对路径的转换和解析。一定程度上放宽了路径格式的限制。

  • 相关阅读:
    随笔
    洛谷
    洛谷
    洛谷
    (水题)洛谷
    洛谷
    (水题)洛谷
    洛谷
    (水题)洛谷
    (水题)洛谷
  • 原文地址:https://www.cnblogs.com/suanec/p/9878439.html
Copyright © 2020-2023  润新知