可映射的CSV读取引擎

由于许多项目都会使用csv文件来存储数据，因此我在这里介绍一套我自己设计出来的解决方案。有不合理的地方还望指出。

一般的csv文件读取都会比较繁琐：按照分隔符(默认逗号)分好行，列，再根据对应的顺序，一行一行，一条一条地读取数据。这本书没什么问题，然而一旦更改csv里的列顺序，或者增删某行就会产生牵一发动全身的结果。而且字段多的时候，写起来是非常反人类的。。

我们项目起初就是用的这种原始解决方案，也的确碰到了上面提及的尴尬局面。后来我想到，如果我能对csv表结构做好映射，像json，像xml那样，不就能大大提高效率？

于是我引出了如下的设计方案：

1. 准备

首先定义两条特性，一条是表整体结构相关的，一条是用来做字段映射

/// <summary>
    /// CSV column mapping
    /// </summary>
    [AttributeUsage(AttributeTargets.Property | AttributeTargets.Field)]
    public class CSVColumnAttribute : Attribute
    {
        /// <summary>
        /// Name of this property/field in csv file(default is property name)
        /// </summary>
        public string Key { get; set; }

        /// <summary>
        /// Column of this property/field in csv file(if column is assigned, key will be ignored)
        /// </summary>
        public int Column { get; set; }

        /// <summary>
        /// Default value(if reading NULL or failed;deault: -1 for number value, null for class, false for bool)
        /// </summary>
        public object DefaultValue { get; set; }

        /// <summary>
        /// Separator for parsing if it's an array(',' by default)
        /// </summary>
        public char ArraySeparator { get; set; }


        public CSVColumnAttribute()
        {
            Column = -1;
            ArraySeparator = '#';
        }

        public CSVColumnAttribute(string key)
        {
            Key = key;
            Column = -1;
            ArraySeparator = '#';
        }

        public CSVColumnAttribute(int column)
        {
            Column = column;
            ArraySeparator = '#';
        }
    }

/// <summary>
    /// CSV Mapping class or struct(Try avoid struct as possible. Struct is boxed then unboxed in reflection)
    /// </summary>
    [AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct)]
    public class CSVMapperAttribute : Attribute
    {
        /// <summary>
        /// Path of the CSV file(without file extension). Base directory is Assets/Resources/
        /// </summary>
        public string Path { get; set; }

        /// <summary>
        /// Mapping key row(0 by default)
        /// </summary>
        public int KeyRow { get; set; }

        /// <summary>
        /// Description row(1 by default. Will be skipped in decoding. If no desc in file, assign -1)
        /// </summary>
        public int DescRow { get; set; }

        /// <summary>
        /// Separator for csv parsing(',' by default)
        /// </summary>
        public char Separator { get; set; }

        /// <summary>
        /// Starting index of data rows
        /// </summary>
        public int StartRow { get; set; }

        public CSVMapperAttribute()
        {
            KeyRow = 0;
            DescRow = 1;
            Separator = ',';
        }

        public CSVMapperAttribute(string name)
        {
            Path = name;
            KeyRow = 0;
            DescRow = 1;
            Separator = ',';
        }
    }

表相关特性里的属性包括：CSV所在路径(可选)，键值所在行(针对非英文表格)，描述所在行(可选)，分隔符(默认为逗号',')，起始行(可选，解析时会跳过这之前的行)

字段映射相关特性的属性包括：键值，对应列号(键值和列号2选1即可。都不设置则默认键值为属性名)，默认值(可选，字段解析失败会返回此默认值)，数组分隔符(可选，默认为'#'，用来分隔数组)

CSVMapperAttribute可以添加到类或结构体上，CSVColumnAttribute可以添加到属性或字段上。

2. 读取和解析

public class CSVEngine
    {
        private List<List<string>> _records;

        /// <summary>
        /// Get column count
        /// </summary>
        public int ColumnCount { get; private set; }

        /// <summary>
        /// Get row count
        /// </summary>
        public int RowCount { get; private set; }

        /// <summary>
        /// Get separator
        /// </summary>
        public char Separator { get; private set; }

        private int _keyRow = -1;
        private int _descRow = -1;
        private int _startRow = -1;

        /// <summary>
        /// Decode CSV file to target mapped type.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="path"></param>
        /// <returns></returns>
        public IEnumerable<T> Decode<T>() where T : new()
        {
            if (_records == null || _keyRow < 0 || _descRow < 0 || _startRow < 0)
            {
                Debug.LogError(string.Format("Decoding Failed: {0}", typeof (T)));
                yield break;
            }

            //Decode each row
            for (int i = _startRow; i < _records.Count; i++)
            {
                if (i == _keyRow || i == _descRow)
                    continue;
                yield return DecodeRow<T>(_records[i], _records[_keyRow]);
            }
        }

        /// <summary>
        /// Decode single row
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="fields"></param>
        /// <param name="keys"></param>
        /// <returns></returns>
        private T DecodeRow<T>(List<string> fields, List<string> keys) where T : new()
        {
            T result = new T();
            IEnumerable<MemberInfo> members =
                typeof (T).GetMembers()
                    .Where(m => m.MemberType == MemberTypes.Property || m.MemberType == MemberTypes.Field)
                    .Where(m => Attribute.IsDefined(m, typeof (CSVColumnAttribute), false));

            if (typeof (T).IsValueType)
            {
                object boxed = result;
                foreach (MemberInfo member in members)
                {
                    CSVColumnAttribute attribute =
                        member.GetCustomAttributes(typeof (CSVColumnAttribute), false).First() as CSVColumnAttribute;
                    string field = GetRawValue(attribute, fields, keys, member.Name);
                    if (ReferenceEquals(field, member.Name))
                        return result;
                    SetValue(member, boxed, field, attribute.DefaultValue, attribute.ArraySeparator);
                }
                return (T) boxed;
            }

            foreach (MemberInfo member in members)
            {
                CSVColumnAttribute attribute =
                    member.GetCustomAttributes(typeof (CSVColumnAttribute), false).First() as CSVColumnAttribute;
                string field = GetRawValue(attribute, fields, keys, member.Name);
                if (ReferenceEquals(field, member.Name))
                    return result;
                SetValue(member, result, field, attribute.DefaultValue, attribute.ArraySeparator);
            }
            return result;
        }

        /// <summary>
        /// Get raw value by CSVColumnAttribute or name
        /// </summary>
        /// <param name="attribute"></param>
        /// <param name="fields"></param>
        /// <param name="keys"></param>
        /// <param name="name"></param>
        /// <returns></returns>
        private string GetRawValue(CSVColumnAttribute attribute, List<string> fields, List<string> keys, string name)
        {
            if (attribute.Column >= 0 && fields.Count > attribute.Column)
            {
                return fields[attribute.Column];
            }
            if (!string.IsNullOrEmpty(attribute.Key) && keys.Contains(attribute.Key))
            {
                return fields[keys.IndexOf(attribute.Key)];
            }
            if (keys.Contains(name))
            {
                return fields[keys.IndexOf(name)];
            }
            Debug.LogError(string.Format("Mapping Error! Column: {0}, Key: {1}, Name:{2}", attribute.Column,
                attribute.Key ?? "NULL", name));
            return name;
        }

        /// <summary>
        /// Parse and set raw value
        /// </summary>
        /// <param name="member"></param>
        /// <param name="obj"></param>
        /// <param name="value"></param>
        /// <param name="defaultValue"></param>
        /// <param name="arraySeparator"></param>
        private void SetValue(MemberInfo member, object obj, string value, object defaultValue, char arraySeparator)
        {
            if (member.MemberType == MemberTypes.Property)
            {
                (member as PropertyInfo).SetValue(obj,
                    ParseRawValue(value, (member as PropertyInfo).PropertyType, defaultValue, arraySeparator),
                    null);
            }
            else
            {
                (member as FieldInfo).SetValue(obj,
                    ParseRawValue(value, (member as FieldInfo).FieldType, defaultValue, arraySeparator));
            }
        }

        /// <summary>
        /// Parse string value to specified type
        /// </summary>
        /// <param name="field"></param>
        /// <param name="type">If type is collection, use array only(e.g. int[])</param>
        /// <param name="defaultValue">If type is collection, use element default(e.g. 0 for int[])</param>
        /// <param name="arraySeparator"></param>
        /// <returns></returns>
        private object ParseRawValue(string field, Type type, object defaultValue, char arraySeparator)
        {
            try
            {
                if (type.IsArray)
                {
                    IEnumerable<object> result =
                        field.Split(arraySeparator)
                            .Select(f => ParseRawValue(f, type.GetElementType(), defaultValue, arraySeparator));
                    if (type.GetElementType() == typeof (string))
                    {
                        return result.Cast<string>().ToArray();
                    }
                    if (type.GetElementType() == typeof (int))
                    {
                        return result.Cast<int>().ToArray();
                    }
                    if (type.GetElementType() == typeof (float))
                    {
                        return result.Cast<float>().ToArray();
                    }
                    if (type.GetElementType() == typeof (double))
                    {
                        return result.Cast<double>().ToArray();
                    }
                    if (type.GetElementType() == typeof (bool))
                    {
                        return result.Cast<bool>().ToArray();
                    }
                    return null;
                }
                if (type == typeof (string))
                {
                    return field;
                }
                if (type == typeof (int))
                {
                    return Convert.ToInt32(field);
                }
                if (type == typeof (float))
                {
                    return Convert.ToSingle(field);
                }
                if (type == typeof (double))
                {
                    return Convert.ToDouble(field);
                }
                if (type == typeof (bool))
                {
                    if (field == null)
                    {
                        return false;
                    }
                    field = field.Trim();
                    return field.Equals("true", StringComparison.CurrentCultureIgnoreCase) || field.Equals("1");
                }
            }
            catch (FormatException ex)
            {
                Debug.LogWarning(string.Format("{0}: {1} -> {2}", ex.Message, field, type));

                //In case default value is null but the property/field is not a reference type
                if (defaultValue == null)
                {
                    if (type == typeof (int) || type == typeof (float) || type == typeof (double))
                    {
                        defaultValue = -1;
                    }
                    else if (type == typeof (bool))
                    {
                        defaultValue = false;
                    }
                }
            }

            return defaultValue;
        }

        /// <summary>
        /// Load CSV into record list. If you need to decode records, use Decode(path) instead.
        /// </summary>
        /// <param name="path"></param>
        /// <param name="separator"></param>
        public bool Load(string path, char separator = ',')
        {
            //Dispose records
            ClearRecord();

            if (string.IsNullOrEmpty(path))
            {
                Debug.LogError(string.Format("CSV path not found: {0}", path));
                return false;
            }

            //Read text
            TextAsset asset = Resources.Load<TextAsset>(path);

            if (asset == null)
            {
                Debug.LogError(string.Format("CSV file not found: {0}", path));
                return false;
            }

            string content = asset.text;
            if (string.IsNullOrEmpty(content))
            {
                Debug.LogError(string.Format("CSV file content empty: {0}", path));
                return false;
            }

            Separator = separator;
            _records = new List<List<string>>();
            foreach (string row in content.Split('
').Where(line => !string.IsNullOrEmpty(line.Trim())))
            {
                List<string> columns = row.Split(separator).Select(s => s.Trim()).ToList();
                //Check each row's column count. They must match
                if (ColumnCount != 0 && columns.Count != ColumnCount)
                {
                    Debug.LogError(
                        string.Format("CSV parsing error in {0} at line {1} : columns counts do not match! Separator: '{2}'", path,
                            content.IndexOf(row), separator));
                    return false;
                }
                ColumnCount = columns.Count;
                _records.Add(columns);
            }
            RowCount = _records.Count;

            if (_records == null || !_records.Any())
            {
                Debug.LogWarning(string.Format("CSV file parsing failed(empty records): {0}", path));
                return false;
            }

            return true;
        }

        public bool Load<T>()
        {
            ClearRecord();

            //Check mapping
            if (!Attribute.IsDefined(typeof (T), typeof (CSVMapperAttribute), false))
            {
                Debug.LogError(string.Format("CSV mapping not found in type: {0}", typeof (T)));
                return false;
            }

            CSVMapperAttribute mapper =
                Attribute.GetCustomAttribute(typeof (T), typeof (CSVMapperAttribute), false) as CSVMapperAttribute;
            _keyRow = mapper.KeyRow;
            _descRow = mapper.DescRow;
            _startRow = mapper.StartRow;

            bool result = Load(mapper.Path, mapper.Separator);
            if (result)
            {
                if (_records[_keyRow].Any(string.IsNullOrEmpty))
                {
                    Debug.LogError(
                        string.Format("Encoding Error! No key column found. Make sure target file is in UTF-8 format. Path: {0}",
                            mapper.Path));
                    return false;
                }
            }
            return result;
        }

        /// <summary>
        /// Get string value at specified row and column. If record empty or position not found, NULL will be returned. Row/Column starts at 0
        /// </summary>
        /// <param name="row"></param>
        /// <param name="column"></param>
        /// <returns></returns>
        public string this[int row, int column]
        {
            get
            {
                if (_records == null || _records.Count <= row || _records[row].Count <= column)
                {
                    return null;
                }
                return _records[row][column];
            }
        }

        /// <summary>
        /// Get a converted value at specified row and column. If record empty or position not found or convertion failed, defaultValue will be returned. Row/Column starts at 0
        /// </summary>
        /// <typeparam name="T">If T is collection, use array only(e.g. int[])</typeparam>
        /// <param name="row"></param>
        /// <param name="column"></param>
        /// <param name="defaultValue">If T is collection, use element default(e.g. 0 for int[])</param>
        /// <param name="arraySeparator"></param>
        /// <returns></returns>
        public T Read<T>(int row, int column, object defaultValue, char arraySeparator = '#')
        {
            string field = this[row, column];
            if (field == null)
            {
                Debug.LogWarning("Field is null. Make sure csv is loaded and field has content.");
                return typeof (T).IsArray ? default(T) : (T) defaultValue;
            }

            return (T) ParseRawValue(field, typeof (T), defaultValue, arraySeparator);
        }


        /// <summary>
        /// Remove all records.
        /// </summary>
        public void ClearRecord()
        {
            _records = null;
        }
    }

看起来比较复杂？我们用例子来讲解：

添加一个表结构类

[CSVMapper("Configs/Resource")]
    public class ResourceData : Data
    {
        [CSVColumn(0)] public int ID;
        [CSVColumn(1)] public string Path;
        [CSVColumn(2)] public float Ratio;
        [CSVColumn(3)] public string Desc;
    }

添加一个根据结构类读表的方法

/// <summary>
    /// Get table
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <returns></returns>
    private IEnumerable<T> GetTable<T>() where T : Data, new()
    {
        CSVReaderX reader = new CSVReaderX();
        if (reader.Load<T>())
        {
            Debug.Log(string.Format("{0} Loaded", typeof (T)));
            return reader.Decode<T>();
        }

        return null;
    }

注意，这里让ResourceData继承Data，并且在GetTable里做了泛型约束是为了规范使用，并无其他意义

Data结构如下

/// <summary>
    /// All table class must inherit this for constraint
    /// </summary>
    public abstract class Data
    {
    }

Resource.csv的内容如下：

资源ID,资源路径,缩放比例,说明
int,string,float,string
10001,Model/a,1,
10002,Model/b,1,
10003,Model/c,1,
10004,Model/d,1,
10005,Model/e,1,
10006,Model/f,1,
10007,Model/g,1,

还可以直接用键值索引：

[CSVMapper("Configs/Resource")]
    public class ResourceData : Data
    {
        [CSVColumn(“资源ID”)] public int ID;
        [CSVColumn(“资源路径”)] public string Path;
        [CSVColumn(“缩放比例”)] public float Ratio;
        [CSVColumn(“说明”)] public string Desc;
    }

第二行(int,string,float,string)其实没什么意义，因此他被当作Desc行(描述行)。

使用延迟实例化加载表格并存储为字典，即可进行键值索引

public Dictionary<int, ResourceData> ResourceDict
    {
        get
        {
            return _resourceDict ?? (_resourceDict = GetTable<ResourceData>().ToDictionary(k => k.ID));
        }
    }

var data = ResourceDict[0];

以上是映射好表结构后自动加载的结果。

我还额外提供了手动解析的接口：

手动Load

public bool Load(string path, char separator = ',');

手动Read

public T Read<T>(int row, int column, object defaultValue, char arraySeparator = '#');

或者通过索引器获得string类型的值再自己解析

        CSVReaderX reader = new CSVReaderX();

        reader.Load("Path");
        int val = reader.Read<int>(0, 0, 0);
        int[] vals = reader.Read<int[]>(0, 0, null);
        string raw = reader[0, 0];

注意，行和列都是从0开始算。

路径因为我这里是Unity3D的项目，所以映射的路径是Resources下不带后缀的路径，且Load方法里用的是Resources.Load方式来读取资源。其他平台的项目做相应修改即可~

集合字段只能用逗号之外的分隔符(默认'#')，且只能为数组类型

    [CSVMapper("Configs/Skill")]
    public class SkillData : Data
    {
        [CSVColumn(0)] public int ID;
        [CSVColumn(1)] public int Name;
        [CSVColumn(2)] public int[] SkillIDs;
    }

有问题欢迎探讨。

源码参见我的github：

https://github.com/theoxuan/MTGeek/blob/master/Assets/Scripts/CSVReaderX.cs