《The Rise and Fall of Scala》scala的兴衰

英文原文：

Five years ago, Scala seemed like the next big thing in programming languages because it elegantly enabled functional programming within an object-oriented paradigm. Today, Scala’s popularity seems to be fading, with companies like LinkedIn and Yammer moving away from it. The TIOBE index (www.tiobe.com) of software language popularity ranked Scala at #13 in 2012; now it’s fallen to #32 in August 2016, being used by less than .6% of the programming community.

Here’s another ominous sign for Scala: Lightbend, its parent company, is now releasing new frameworks with a Java API before the Scala version. Anecdotally, as CTO of a leading software product engineering company, I meet many software development managers, and I know of at least two who have made the painful decision to abandon Scala after more than a year of adoption. What happened? What gave Scala its initial popularity boost, and what caused its decline? Are there any use cases for which Scala is still the best choice?

To understand the initial popularity of Scala, we must first understand the evolution of modern programming paradigms. First, there was procedural programming, where programs were viewed as a series of statements that should be executed one after another. Then came object-oriented programming, where programs were viewed as actors that knew how to perform operations on objects and converse with each other in order to accomplish tasks.

Functional programming, by contrast, views a program as a mathematical function which is evaluated to produce a result value. That function may call upon nested functions, which in turn may call upon more nested functions. A nested function evaluates to produce a result. From there, that result is passed on to the enclosing function, which uses the nested function values to calculate its own return value. To enable functions to easily pass data to and from other functions, functional programming languages typically define data structures in the most generic possible way, as a collection of (any) things. They also allow functions to be passed to other functions as if they were data parameters. A function in this paradigm is not allowed to produce any side effects such as modifying a global variable that maintains state information. Instead, it is only allowed to receive parameters and perform some operations on them in order to produce its return value. Executing a functional program involves evaluating the outermost function, which in turn causes evaluation of all the nested functions, recursively down to the most basic functions that have no nested functions.

Why is functional programming a big deal?

• Clarity: Programming without side effects creates code that is easier to follow – a function is completely described by what goes in and what comes out. A function that produces the right answer today will produce the right answer tomorrow. This creates code that is easier to debug, easier to test, and easier to re-use.
• Brevity: In functional languages, data is implicitly passed from a nested function to its parent function, via a general-purpose collection data type. This makes functional programs much more compact than those of other paradigms, which require substantial “housekeeping” code to pass data from one function to the next.
• Efficiency: Because functions do not have side effects, operations can be re-ordered or performed in parallel in order to optimize performance, or can be skipped entirely if their result is not used by any other function.

Functional programming languages have existed for decades, beginning with John McCarthy’s LISP language created at MIT in the 1950s. However, these were always viewed as niche languages, of interest mainly to academicians and theoreticians. Scala also began as an academic project, created in 2001 at the Ecole Polytechnique Federale de Lausanne by Martin Odersky. However, Scala designers made several critical decisions that positioned Scala to be the breakthrough language that would bring functional programming to the mainstream:

• Scala combines functional programming with object-oriented programming. As a multi-paradigm language, it can serve as a bridge for object-oriented programmers entering the world of functional programming.
• Scala code runs in the Java Virtual Machine (JVM). This means it can be readily deployed on any machine that runs Java (around 85% of PC’s). It also means Scala code can theoretically interoperate with Java code, providing a bridge for a Java development team to ease into Scala.
• Scala is syntactically similar to Java, and, like Java, performs type-checking at compilation time rather than at runtime, thus eliminating the possibility of runtime errors caused by type incompatibility. These similarities reduce the initial learning curve for Java programmers.
• Scala has built-in support for pattern matching, where arbitrary data types can be matched based on value patterns, in order to perform distinct operations on each matched pattern.
• Scala includes Akka as a standard library, enabling support for rich concurrency models. This makes it easy for programmers to implement complex creation or processing of streaming data as a graph of actors, where each actor processes data in parallel.

It’s no wonder then that Scala was initially greeted with such enthusiasm and was viewed as the language that would bring functional programming to the mainstream. Yet in the words of William H. Calvin, “You can always spot the pioneers by the arrows in their backs.” Scala was surely a pioneer in popularizing functional programming. So why now has the tide turned on Scala, to the point where today its developer base is steadily shrinking?

• The Java programming language introduced functional programming constructs beginning with Java 8, released in early 2014. There are subtle differences in the ways Scala and Java support functional programming, and the argument can be made that Scala’s approach is superior. But, Java has surpassed Scala as the preeminent functional programming language, because programmers already know Java. This is reminiscent of the arc of Adobe Flex and Microsoft Silverlight, which had a sizable following among Web UI programmers, until the release of HTML 5, which provided enough Web UI features to become the dominant technology.
• Scala is a difficult language to master because its principles are based on mathematical type theory, which is fully understood by only the most academic and mathematically minded programmers. Furthermore, many language features of Scala, including implicits and macros, may cause the control of the program to flow unexpectedly to other parts of the code base, which makes it difficult for most programmers to follow or debug their code. An excellent programmer who comprehends all this will be more productive in Scala than in Java, but an average programmer’s productivity, as measured by implemented functionality, will probably decline when transitioning from Java to Scala. This is not just a short-term decline due to a learning curve — the decline has been observed in some development teams a full year after adopting Scala.
• Unlike Java, Scala has a flexible syntax, and typically offers many ways to achieve the same end result. Rather than making Scala more usable for average programmers, the Scala community seems to spend a lot of time arguing about which of several functionally equivalent solutions is the right one. These debates generate more heat than light, and prevent the emergence of tried-and-true implementation patterns that exist in other, more restrictive languages like Java.
• Scala has not done a good job of maintaining compatibility, either with earlier versions of Scala or with Java.

Because of these issues, Scala will likely never evolve into a mainstream programming language like Java. However, there are still specific use cases where Scala is such an ideal fit that it should be the programming language of choice:

• Big Data manipulation: Scala’s strengths are highly suited to the Big Data programming model, where a task takes an immutable collection as input, transforms the collection with map and reduce operations, and generates a new result collection. For Big Data tools such as Spark, the advantages of using Scala are overwhelming, given that a Scala version of a program will usually be five-10 times shorter than the equivalent Java program.
• Creating domain-specific languages: Many problems are best solved by giving users a domain-specific language (DSL) that they can use to write scripts. For example, suppose your users are asking for a tool that will enable them to schedule and run automated QA tests. Since you cannot predict all the combinations of tests they will need to run, you need to give them a scripting language where they can define the order and location of these tests for any scenario. Scala is uniquely suited for development of DSLs, thanks to features such as pattern matching, syntactic flexibility, and operator overloading.

To summarize: Scala played a key role as a catalyst in popularizing functional programming, and it exerted a strong influence on the design of functional programming in Java. Scala will probably never become the next big programming language. But, it will be around for years to come as the language of choice for niche problem domains such as Big Data programming.

译文：

5年前，Scala 似乎曾要成为编程语言中下一个佼佼者，因为它能够优雅得使用面向对象编程范式进行函数编程。

现如今，随着像 LinkedIn 和 Yammer 这些公司的弃用，Scala 的光环正逐渐黯淡。

2012 年的 TIOBE 编程语言受欢迎度排行榜上，Scala 排名第 13 位；2016 年 8 月竟下降到第 32 位，现在只有不到 6% 的编程社区在使用它。

不祥的预兆：Lightbend，Scala 的母公司，在先前的 Scala 版本上发布了一款使用 Java API 的新框架。

有趣的是，作为一家领先的软件产品工程公司的 CTO，我见了很多软件开发主管，我知道的至少有两个人，曾经在使用了 Scala 一年多后，便痛苦的决定放弃使用它。这是为什么呢？

最初是什么给了 Scala 如此高的知名度呢，而如今又是什么导致了它的衰退呢？

有没有一些案例，能够证明使用 Scala 仍然是最好的选择呢？

想要知道 Scala 最初是如何火起来的，就要先了解现代编程范式的演变。首先，程序化编程，程序被视为是应该被一个接一个执行的一系列声明。然后是面向对象编程，知道如何执行对象上的操作，以及如何与相互交流，从而完成任务。

相比之下，函数式编程将一个程序作为数学函数来评估，以生成一个结果值。该函数可以调用嵌套函数，而嵌套函数又可以调用更多的嵌套函数。一个嵌套函数求出一个结果。然后，该结果会被传递给封闭的函数，这是使用嵌套函数值来计算它自己的返回值。为了使函数能够方便传递数据，并且从其他函数中，函数编程通常作为一个集合，以最可能的方式定义数据结构。它们还允许函数间传递，就像它们是数据参数一样。在这个参数内的一个例子是不允许产生任何副作用，像修改一个全局变量保持状态信息。相反，它只允许接收参数，并且为了生成其返回值，会对它们进行一些操作。执行一个函数程序包括评估最外层的功能，这反过来又导致了对所有的嵌套函数评估，而最基本的功能递归向下是没有嵌套功能的。

为什么函数式编程如此重要？

1. 清晰：没有副作用的编程能创建出更容易有规律可循的代码 - 一个函数通过输入和输出完全被描述出来。一个函数今天可以生成正确答案，明天也会生成正确答案。这样创建的代码更容易调试，测试和重用。
2. 简洁：在函数语言中，数据通过通用集合数据类型从嵌套函数隐式传递到其父函数。
3. 高效：由于函数没有副作用，运算可以重新排序或并行执行，以优化其性能，或者，如果它们的结果没有被其他任何函数使用，则可以完全跳过。

函数编程语言已经存在了几十年，开始于 John McCarthy 的 LISP 语言，这是20世纪50年代在麻省理工学院创建的。然而，这些总是被视为细分语言，感兴趣的主要是院士和理论家。Scala 开始还作为一个学术项目，2001年由 Martin Odersky 创建于 Ecole Polytechnique Federale de Lausanne。随之，Scala 的设计者做出了几个重要决定，将 Scala 定位为函数编程到主流的突破性语言。

• Scala 代码在 Java 虚拟机（JVM）中运行。这意味着它可以轻松部署在任何运行 Java（大约85%的PC）的机器上。还意味着，Scala 代码在理论上可以与 Java 代码相互操作，为 Java 开发团队提供了一个桥梁，以轻松进入 Scala。
• Scala 在语法上和 Java 相似，并且像 Java 一样，在编译时执行类型检查而不是在运行时，从而消除了由类型不兼容而导致运行错误的可能性。这些相似之处减少了 Java 程序员的初始学**曲线。
• Scala 内置对模式匹配的支持，可以基于值模式来匹配任意数据类型，以便于对每个匹配模式执行不同的操作。
• Scala 将 Akka 作为一个标准库，支持丰富的并发模型。这使得程序员很容易的就可以实现流数据的复杂创建或处理。

看到这里，也难怪 Scala 当时那么受欢迎，被视为引领函数编程的主流语言。然而用 William H. Calvin 的话来说，“你总是可以通过背后的箭头发现先驱者。”毫无疑问，Scala 便是函数编程得以普及的先驱。

那么，为什么现在的趋势又转向 Scala 了呢，今天它的开发者们是又如何到了稳步缩减的地步？

• Java 编程语言引入了函数式编程结构，始于2014年初发布的 Java 8。Scala 和 Java 支持的函数编程方式有着微妙的差别，有争议的是，Scala 的方法更胜一筹。但是，作为优秀的函数编程语言，Java 已经超越了Scala，因为程序员们对 Java 已经很了解了。这让人联想到 Adobe Flex 和 Microsoft Silverlight，它在 Web UI 程序员中有相当大的跟风性，直到发布了 HTML 5，HTML 5 提供了足够的 Web UI 特性，使其成为主流技术。
• Scala 是一门很难掌握的语言，因为它的规则是基于数学类型理论下的，只有最具学术和数学天分的程序员才能够完全理解。此外，Scala 的很多语言特性，包括隐性和宏，可能导致程序控制意外流到代码库的其他部分，这使得大多数程序员难以跟踪或调试其代码。一个能够领会这一切的优秀程序员，使用 Scala 会比使用 Java 更高效，但是一个普通程序员的生产力，从功能实现上来看，效率则会相反。这不仅仅是由于学**曲线的短期下降 — 这是一些开发团队使用 Scala 一整年后的观察结果。
• 不像 Java，Scala有一个灵活的语法，通常会提供很多方法来达到相同的结果。除了让 Scala 为更多的程序员所用外，Scala 社区似乎花了很多时间来讨论，这个几个功能解决方案哪个是正确的。这些争论聚集了很高的热度，并且防止出现在其他更具限制性的语言（如Java）中存在的尝试和真正的实现模式。
• Scala 还没有做好保持兼容性的工作，无论是早期版本的 Scala 还是 Java。

鉴于这些问题，Scala 很可能永远不会演变成 Java 这样的主流编程语言，但仍然会有一些特殊案例，选择 Scala 这种编程语言才是最合适的：

• 大数据处理：Scala 的优势非常适合大数据编程模型，其中任务采用了不可变集合作为输入，使用 map 和 reduce 处理转换集合，并生成新的结果集合。对于 Spark 等大数据工具，使用 Scala 的优势是压倒性的，因为一个程序的 Scala 版本通常比等效的 Java 程序缩短 5 到 10 倍。
• 创建特定领域的语言：通过为用户提供可用于编写脚本的特定领域的语言（DSL），可以很好的解决许多问题。例如，假设你的用户要求用一个工具，使他们能够安排和运行自动化质量检测。由于你不能预测所有需要运行的测试组合，因此你需要为他们提供一种脚本语言，以便于他们为任何场景定义这些测试的顺序和位置。由于其模式匹配，语法灵活，操作符重载等特性，使 Scala 能够独特地适用于 DSLs 开发。

总结

Scala 在函数编程普及的过程中起到了关键性的催化剂作用，它对 Java 中的函数式编程设计产生了很大的影响。Scala 可能永远不会成为下一个巨星编程语言，但是，在未来几年内，它将成为大数据编程等细分问题领域的首选语言。