Model persistence of sklearn

Model persistence

https://scikit-learn.org/stable/modules/model_persistence.html

     模型训练完毕后，如何保存起来，以便日后使用呢？这就是模型持久化。

After training a scikit-learn model, it is desirable to have a way to persist the model for future use without having to retrain. The following sections give you some hints on how to persist a scikit-learn model.

Python specific serialization

sklearn是基于python语言， python本身支持对象的序列化功能。

第一个方法就是使用pickle，将模型进行序列化 和 反序列化。

It is possible to save a model in scikit-learn by using Python’s built-in persistence model, namely pickle:

>>> from sklearn import svm
>>> from sklearn import datasets
>>> clf = svm.SVC()
>>> X, y= datasets.load_iris(return_X_y=True)
>>> clf.fit(X, y)
SVC()

>>> import pickle
>>> s = pickle.dumps(clf)
>>> clf2 = pickle.loads(s)
>>> clf2.predict(X[0:1])
array([0])
>>> y[0]
0

第二个方法使用 joblib库的 dump 和load接口， 此工具比pickle更加高效， 但是它只支持保存为文件， 不能转换为二进制串。

In the specific case of scikit-learn, it may be better to use joblib’s replacement of pickle (dump & load), which is more efficient on objects that carry large numpy arrays internally as is often the case for fitted scikit-learn estimators, but can only pickle to the disk and not to a string:

>>> from joblib import dump, load
>>> dump(clf, 'filename.joblib') 

Later you can load back the pickled model (possibly in another Python process) with:

>>> clf = load('filename.joblib') 

Security & maintainability limitations

python的序列化方法，有缺点：

（1）安全性问题： 对于一个不可信的pickle文件，如果含有恶意代码， 加载到内存产生安全隐患。

（2）可维护性问题： 如果训练环境的sklearn 和 加载环境的sklearn 不是相同的版本， 有可能不支持。

为了解决这个可维护性问题， 一般将训练环境的做成docker容器， 在部署过程中，会将模型在此docker容器中加载，并运用。

pickle (and joblib by extension), has some issues regarding maintainability and security. Because of this,

• Never unpickle untrusted data as it could lead to malicious code being executed upon loading.

• While models saved using one version of scikit-learn might load in other versions, this is entirely unsupported and inadvisable. It should also be kept in mind that operations performed on such data could give different and unexpected results.

In order to rebuild a similar model with future versions of scikit-learn, additional metadata should be saved along the pickled model:

• The training data, e.g. a reference to an immutable snapshot

• The python source code used to generate the model

• The versions of scikit-learn and its dependencies

• The cross validation score obtained on the training data

This should make it possible to check that the cross-validation score is in the same range as before.

Since a model internal representation may be different on two different architectures, dumping a model on one architecture and loading it on another architecture is not a supported behaviour, even if it might work on some cases. To overcome the issue of portability, pickle models are often deployed in production using containers, like docker.

If you want to know more about these issues and explore other possible serialization methods, please refer to this talk by Alex Gaynor.

Interoperable formats

    对于上面问题，实际上是面向模型在不同架构和平台上的互操作性， 工业界提出两种解决方案：

（1） Open Neural Network Exchange 一个模型的二进制序列化规范和工具。

（2） Predictive Model Markup Language (PMML)  一个机遇XML格式的模型表示规范。 具有可读性， 便于在不同框架或者平台上测试相同模型配置的性能。

For reproducibility and quality control needs, when different architectures and environments should be taken into account, exporting the model in Open Neural Network Exchange format or Predictive Model Markup Language (PMML) format might be a better approach than using pickle alone. These are helpful where you may want to use your model for prediction in a different environment from where the model was trained.

ONNX is a binary serialization of the model. It has been developed to improve the usability of the interoperable representation of data models. It aims to facilitate the conversion of the data models between different machine learning frameworks, and to improve their portability on different computing architectures. More details are available from the ONNX tutorial. To convert scikit-learn model to ONNX a specific tool sklearn-onnx has been developed.

PMML is an implementation of the XML document standard defined to represent data models together with the data used to generate them. Being human and machine readable, PMML is a good option for model validation on different platforms and long term archiving. On the other hand, as XML in general, its verbosity does not help in production when performance is critical. To convert scikit-learn model to PMML you can use for example sklearn2pmml distributed under the Affero GPLv3 license.

ONNX

https://onnx.ai/supported-tools.html#buildModel

The ONNX community provides tools to assist with creating and deploying your next deep learning model. Use the information below to select the tool that is right for your project.

Frameworks & Converters

Use the frameworks you already know and love.

 

PMML

http://dmg.org/pmml/v4-4-1/GeneralStructure.html

PMML Version	Model Type	Vendor	Application	Dataset	PMML File
4.1	Clustering	KNIME	KNIME 2.8	Audit	View
4.1	Clustering	KNIME	KNIME 2.8	Iris	View
4.1	Neural Network	KNIME	KNIME 2.8	Audit	View
4.1	NeuralNetwork	KNIME	KNIME 2.8	Iris	View
4.1	Regression	KNIME	KNIME 2.8	Audit	View
4.0	Regression	KNIME	KNIME 2.6.2	Elnino	View
4.0	Regression	KNIME	KNIME 2.6.2	Elnino	View
4.1	Regression	KNIME	KNIME 2.8	Iris	View
4.1	Tree	KNIME	KNIME 2.8	Audit	View
4.1	Tree	KNIME	KNIME 2.8	Iris	View
4.1	Support Vector Machine	KNIME	KNIME 2.8	Audit	View
4.1	Support Vector Machine	KNIME	KNIME 2.8	Iris	View
4.1	Model Ensemble - Clustering	KNIME	KNIME 2.8	Audit	View
4.1	Model Ensemble - Neural Network	KNIME	KNIME 2.8	Audit	View
4.1	Model Ensemble - Neural Network	KNIME	KNIME 2.8	Iris	View
4.1	Model Ensemble - Regression	KNIME	KNIME 2.8	Audit	View
4.1	Model Ensemble - Regression	KNIME	KNIME 2.8	Iris	View
4.1	Model Ensemble - Tree	KNIME	KNIME 2.8	Audit	View
4.1	Model Ensemble - Tree	KNIME	KNIME 2.8	Iris	View
4.1	Model Ensemble - SVM	KNIME	KNIME 2.8	Audit	View
4.1	Model Ensemble - SVM	KNIME	KNIME 2.8	Iris	View
3.2	Clustering	R/Rattle	PMML Package 1.2.29	Audit	View
3.2	Clustering	R/Rattle	PMML Package 1.2.29	Iris	View
3.2	Clustering	R/Rattle	PMML Package 1.2.29	Iris	View
3.2	Tree	R/Rattle	PMML Package 1.2.29	Audit	View
3.2	Tree	R/Rattle	PMML Package 1.2.29	Iris	View
3.2	Regression	R/Rattle	PMML Package 1.2.29	Audit	View
3.2	Regression	R/Rattle	PMML Package 1.2.29	Iris	View
3.2	Regression	R/Rattle	PMML Package 1.2.29	Iris	View
4.0	Support Vector Machine	R/Rattle	PMML Package 1.2.30	Audit	View
4.0	Random Forest	R/Rattle	PMML Package 1.2.30	Audit	View
4.0	Random Forest	R/Rattle	PMML Package 1.2.30	Iris	View
4.0	General Regression	R/Rattle	PMML Package 1.2.30	Iris	View
4.0	Association Rules	R/Rattle	PMML Package 1.2.30	Shopping	View
4.1	Transformations	R/Rattle	PMML Package 1.3	Audit	View
4.1	Transformations	R/Rattle	PMML Package 1.3	Iris	View
4.2	Clustering	Apache Spark	Apache Spark MLlib 1.4	Iris	View