Friday, October 23, 2015

TL;DW for A Brief, Opinionated History of the API

Josh Bloch's whirlwind talk A Brief, Opinionated History of the API is worth watching, both for its historical exploration that situates our own work and for the clarity it brings regarding this lawsuit between Oracle and Google. The following are some notes I took as I watched.

The beginning (of my notes, and of his talk) is more detailed in order to show the development of the idea.

  1. Driving question: "How did APIs develop in the history of programming?"
  2. Who invented the API?
    1. 00:59 "Subroutine library" gave idea of re-using code for common operations (von Neumann & Goldstine, design for EDVAC, 1948)
    2. 02:26 ACM awarded Turing Award to Maurice Wilkes (1967)
      1. Why not awarded to von Neumann & Goldstine?
    3. 03:16 Wilkes got the credit because von Neumann and Goldstine had an idea, but hadn't implemented it.
      1. Theory --> Practice
    4. 04:47 Wilkes' machine, EDSAC, came online in 1951 and was immediately useful.
    5. 06:40 Wilkes' machine was immediately useful because he prioritized simplicity over performance.
      1. Goal: produce a working machine quickly, rather than to create a more refined machine that would take longer to build.
      2. [Agile process. Win! :)]
    6. 08:45 First programs to run were toys, and their architecture was:
      1. first 30 words were "initial orders" (boot loader)
        1. Pressing start button [heh, 44 years before Windows 95] loaded initial orders to memory and began execution.
      2. program was loaded from tape into memory, starting at location 30
      3. programs were written in assembly
    7. 09:52 Wilkes' first real program: he realized that a good part of the remainder of his life was going to be spent debugging.
      1. Saw subroutines as the solution, assigned it to his grad student, Wheeler.
    8. 11:13 Wheeler architected working subroutines that required no manual intervention within three(!) months.
      1. "coordinating orders" augmented "initial orders"
      2. "pseudo-orders" for relocation of subroutines, parameter assignment, etc.
      3. initial orders ran coordinating orders interpretively
    9. 12:25 subroutine linkage technique allowed for
      1. self-modifying code (good idea when you're working with 512 words of memory, 1024 bytes)
      2. subroutines could invoke other subroutines ad infinitum (recursion didn't come for another decade)
    10. 13:41 categories of subroutines in the EDSAC subroutine library
      1. floating point arithmetic
      2. arithmetic on complex numbers
      3. checking (dynamic debugging!)
      4. division
      5. exponentials
      6. general routines relating to functions
      7. differential equations
      8. special functions
      9. power series
      10. logarithms
      11. misc.
      12. print & layout
      13. quadrature
      14. read (i.e., input)
      15. nth root
      16. trig functions
      17. counting operations
      18. vectors & matrices
    11. 14:37 The Preparation of Programs for an Electronic Digital Computer
      1. world's first computer programming text
      2. definitive until high level languages arose
      3. contained entire API
      4. cited in Wilke's Turing Award
      5. Tech report published: 1950
      6. Published as a book in 1951
    12. 16:08 Wheeler presented key ideas in 1952 paper to ACM; these had all been implemented:
      1. subroutine
      2. subroutine library
      3. generality vs. performance tradeoffs
      4. importance & difficulty of library documentation
      5. information hiding
      6. the interpretive routine (in order to squeeze your program into memory: write your programs in your own optimized mini-language, and then interpret that)
      7. the interpretive debugger
      8. higher-order functions
    13. 18:00 Quote from Wheeler's paper: "It should be pointed out that the preparation of a library sub-routine requires a considerable amount of work. This is much greater than the effort merely required to code the sub-routine in its simplest possible form. It will usually be necessary to code it in the library standard form and this may detract from its efficiency in time and space. It may be desirable to code it in such a manner that the operation is generalized to some extent. However, even after it has been coded and tested there still remains the considerable task of writing a description so that people not acquainted with the interior coding can nevertheless use it easily. This last task may be the most difficult."
    14. 18:35 Conclusion of Wheeler's paper: "The prime objectives to be borne in mind when constructing sub-routine libraries are simplicity of use, correctness of codes and accuracy of description. All complexities should—if possible—be buried out of sight."
    15. 19:41 Wheeler's paper was only two pages long.
    16. 20:12 Why didn't Wilkes and Wheeler discuss the API as distinct from the library?
      1. the two were largely isomorphic
      2. one machine [architecture]; no notion of portability
      3. no legacy programs; no notion of backward compatibility
    17. 20:58 reimplementation of existing subroutine libraries for new hardware and with better algorithms: APIs became independent from libraries
    18. 21:36 Bloch's own research: 1968 paper first to use the term "Application Program Interface"
    19. 23:33 Why did the term arise?
      1. goal of allowing implementations to be replaced without harm to clients
      2. needed a name for the concept
      3. libraries in practice give rise to APIs: APIs were discovered, more than invented
  3. What exactly is an API?
    1. 25:16 Bloch's proposed simple, working definition: "An application programming interface (API) specifies a component in terms of its operations, their inputs, and outputs. Its main purpose is to define a set of functionalities that are independent of their implementation, allowing the implementation to vary without compromising the users of the component."
    2.  25:41 If you can answer yes to these questions, it's an API:
      1. Does it provide a set of operations defined by their inputs and outputs?
      2. Does it admit reimplementation without compromising its users?
    3. 26:40 FORTRAN II standard library, 1958. The API still works in modern FORTRAN. (!!!!!)
      1. API? YES
    4. … Bloch reviews 12 other instruction sets, standard library documents, etc. to see if they pass his two-question API Test.
    5. 31:07 It seems that too many things meet Bloch's definition: Instruction Set Architectures, CLIs, wire-level protocols. Augment definition [augmentation in italics]:
      1. "An application programming interface (API) specifies a component in terms of its operations, their inputs, and outputs. Its main purpose is to define a set of functionalities that are independent of their implementation, allowing the implementation to vary without compromising the users of the component. An API augments a programming language (or set of languages with an interoperable calling convention). Alternatively an API may be described in an interface definition language."
    6. 31:50 Four lessons from quick tour:
      1. APIs come in all shapes and sizes (and keep getting bigger!)
      2. Many APIs live forever (outliving the platforms for which they were created)
      3. APIs can create entire industries above/beneath
      4. APIs are the methods of operation by which components in a system use one another.
  4. How does an API come to be?
    1. 32:49 People write software to address a pain point. If others like your software, they use it. BOOM. Now you have an API, whether you were ready for it, or not.
    2. "Necessity is the mother of the API."
  5. What makes an API successful?
    1. 33:20 Right thing (solves a real problem), right place (successful language or platform), right time, good enough
    2. 34:22 Success isn't everything!
      1. If your API is successful and not good, that's bad. If it's good and not successful, it may influence a successful API in the future.
    3. 34:48 Morals
      1. You can't know which APIs will take off, or when.
      2. Design all interfaces as if they were public APIs. (They might be!)
      3. Don't wait to design in the quality. (It might not be possible, later, and your API may take off while it still sucks.)
      4. Principles of API design are well known, as are the costs of ignoring them.
  6. A legal digression
    1. 35:40 We've always had the freedom to reimplement each others' APIs.
    2. Current (disfavorable) conclusion to the court case Bloch mentions: http://bits.blogs.nytimes.com/2015/06/29/supreme-court-declines-to-hear-appeal-in-google-oracle-copyright-fight/

Thursday, October 15, 2015

TL;DW for Unknown pearls from the Clojure standard library

This brief and to-the-point talk by Renzo Borgatti can be seen here. He goes through 10 relatively unknown fns that are available without external dependencies, and interesting in some way. Here's the list:
  1. destructure—useful to debug destructuring. "It's like macroexpand, but for destructuring."
  2. reductions—like reduce, but also returns intermediate results
  3. test—docstring says it best: "test [v] finds fn at key :test in var metadata and calls it, presuming failure will throw exception". You might use this to document/demonstrate assertions about a var in the immediate context of the var's definition.
  4. clojure.pprint/cl-format—crazy-powerful formatting function from Common Lisp. Pluralization of English words! Roman numerals! Spelled out English representations of numbers! The docstring links to this documentation on format control strings.
  5. clojure.java.browse/browse-url—programmatically open a URL in the system browser
  6. clojure.java.javadoc/javadoc—quick peek into java docs
  7. clojure.reflect/reflect—deep reflection on types. Get variables, fields, methods supported, signatures of each method, etc.
  8. clojure.inspector/inspect-tree—visual inspector of data structures, handy for complex structures. Swing UI.
  9. clojure.lang.PersistentQueue—immutable FIFO queue, with buffers, schedulers, etc.
  10. fnil—nil-patch a fn. Useful to handle nil when it wouldn't be handled otherwise, or else to override nil-handling.
The following get honorable mentions:
  1. counted?—does coll implement 'count in constant time?
  2. reversible?—does coll implement Reversible?
  3. vector-of—uniform-type vectors of unboxed primitives
  4. clojure.set/rename-keys—rename keys in a map
  5. clojure.data/diff—Clojure data structure diffing
  6. munge—munge special characters in symbols or strings to _ENGLISH_ representations. (munge "!") -> "_BANG_"
  7. gensym—Returns a new symbol with a unique name, optionally prefixed.
  8. seque—Creates a queued seq on another (presumably lazy) seq s.
  9. zippersfunctional tree editing
These were highlights for me: destructure, test, cl-format, reflect, inspect-tree, munge