auto ml
关于 auto-ml 韩松的那几篇论文膜拜一下
sysml 的会议target research at the intersection of systems and machine learning. The conference aims to elicit new connections amongst these fields, including identifying best practices and design principles for learning systems, as well as developing novel learning methods and theory tailored to practical machine learning workflows.
Michael Jordon: 深度学习像黑箱子，不知道具体的。。。 dirty work
TVM: I find this field extremely exciting every advance you make in this field can make a real world impact.
solution : computational graph optimization ....
Systems for training and serving machine learning models at scale
摩尔定律已经终结”“设计人员无法再创造出可以实现更高指令级并行性的CPU架构”“晶体管数每年增长50%，但CPU的性能每年仅增长10%”。此次黄仁勋演讲的主题依旧是AI，会上，他先后宣布了阿里巴巴、百度和腾讯已经在云服务中使用上Volta GPU，华为、浪潮、联想等服务 器厂商也部署 了基于HGX的GPU服务器。
芯片尺寸 不再 不断缩小，NVIDIA的GPU可以弥补CPU的不足，加强高强度计算负载,是面向AI等 未来应用场景最理想的方案。
所谓摩尔定律是指芯片上的晶体管数量每隔24个月将增加一倍（即半导体行业产品的性能每两年翻一倍）。
与以往首先改善芯片、软件随后跟上的发展趋势不同，以后半导体行业的发展将首先看软件——从手机到超级电脑再到云端的数据中心——然后反过来看要支持软件和应用的运行需要什么处理能力的芯片来支持，由于新的计算设备变得越来越移动化，新的芯片中，可能会有新的一代的传感器、电源管理电路和其他的硅设备
https://new.qq.com/omn/20180904/20180904A0K4O7.html
然而，Peng 认为更快更好的系统不仅需要通过处理器技术实现，还需要通过架构来实现。系统架构本身也面临着诸多挑战，特别是功率和密度，这也限制了性能。「在本世纪的头十年，这种状况渐渐走到了尽头。从 2010 t年起，计算环境开始向异构系统发展，这时我们的计算机所使用的处理器可以被分为通用处理器以及那些你可以广泛称之为固有硬件加速器的处理器。这样的处理器可能是一个 CPU 或一个 MPU，当然，在机器学习领域 ASIC 也渐渐复兴起来。」机器学习和其他现代计算工作，以及激增的连接起来的智能设备（数百亿），正推动新一轮的对硅技术的投资和对可配置和适应性强的硬件平台的需求。异构结构设计将是推动性能提升的关键。
From CMU Eric Xing:
The rise of Big Data requires complex Machine Learning models with millions to billions of parameters, that promise adequate capacity to digest massive datasets and offer powerful predictive analytics (such as high-dimensional latent features, intermediate representations, and decision functions) thereupon. In turn, this has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters. In order to support the computational needs of ML algorithms at such scales, an ML system often needs to operate on distributed clusters with 10s to 1000s of machines; however, implementing algorithms and writing systems softwares for such distributed clusters demands significant design and engineering effort. A recent and increasingly popular trend toward industrial-scale machine learning is to explore new principles and strategies for either highly specialized monolithic designs for large-scale vertical applications such as various distributed topic models or regression models, or flexible and easily programmable general purpose distributed ML platforms—- such as GraphLab based on vertex programming, and Petuum using parameter server. It has been recognized that, in addition to familiarity of distributed system architectures and programing, large scale ML systems can benefit greatly from ML-rooted statistical and algorithmic insights, which can lead to principles and strategies unique to distributed machine learning programs. These principles and strategies shed lights to the following key questions—- How to distribute an ML program over a cluster? How to bridge ML computation with inter-machine communication? How to perform such communication? What should be communicated between machines?—- and they span a broad continuum from application, to engineering, and to theoretical research and development of Big ML systems and architectures. The ultimate goal of large scale ML systems research is to understand how these principles and strategies can be made efficient, generally-applicable, and easy to program and deploy, while not forgetting that they should be supported with scientifically-validated correctness and scaling guarantees.
==================================
https://blog.algorithmia.com/deploying-machine-learning-at-scale/
================================
Deep learning (DL) heavily relies on fast hardware and parallel algorithms to train complex neural networks. This BoF will bring together researchers and developers working on the design of next-generation computer systems for DL and parallel DL algorithms that can exploit the potential of these new systems. Research teams working on major deep learning systems deployed in the field are invited to discuss latest hardware and software trends and to exchange views on the role Artificial Intelligence (AI) in general and DL in particular will play in the near future for big data analytics and HPC applications.