Gig City Elixir has wrapped up; it was a great conference and I really enjoyed being around so many people excited about the language. I wanted to capture my thoughts on the second day’s talks while they were still fresh in my mind.
Stewardship Made Practical
By Stuart Halloway
Stuart’s talk focused on what it is like to develop OSS on the Internet and some of the negatives associated with that. While he was nominally talking about OSS maintainers, his message resonated with me as a consumer of OSS with good guidelines of what not to do.
Stuart works at Cognitect developing Clojure and framed a lot of this talk with an email sent to the Clojure mailing list by Rich Hickey, establishing guidelines for decorum on the list. The choice quote is “Clojure is run by, and for, people who make things.” This helped focus the community around ideas and treating each other civilly.
Stuart then worked through four aspects of stewardship:
- How To Make Software
- How To Behave
- What To Expect
- Be Good To Yourself
Making maintainable software requires design, growth and simplicity, all of which skills that can be learned. When designing, you should maintain your artifacts, e.g. what is the rationale for a new feature, to help people decide if what you’ve built will work for them. Stuart also made an interesting point on growth, which was to not think of it as “change” but to instead identify which of three types of changes you’re making:
- Accretion, where the system/service/library provides more to the user
- Relaxation, where the system/service/library requires less of the user
- Fixation, where bugs are being addressed
Stuart also made a bold declaration that SemVer is “broken” and should be disposed of, because it’s impossible to tell if your particular usage of a library is impacted when the version changes. Instead, names (or namespaces) should always provide the same functionality, and the names (or namespaces) should be changed, instead of the version, when the functionality changes.
On behavior, newcomers should respect a community’s process, taking the time to learn the behavioral norms before making suggestions. Indeed, Stuart went further and insisted that there is never an appropriate time to suggest that someone else change their workflow (he would later walk this back slightly and admit that saying never was a bit of rhetoric). Finally, people should disagree with civility, always grounding their arguments with facts and logic.
The meat of Stuart’s talk was spent on what to expect when you’re in charge of OSS. He began by citing Sturgeon’s Revelation: “90% of everything is crap.” Maintainers receive a constant stream of suggestions and criticisms, and a decent chunk of those are from people behaving badly. Disagreements typically come in the form of rhetoric, such as arguments to authority and ambiguous collective (referring to some unspecified group that supports or will benefit from the change). There are typically three root causes for these types of comments:
- Thinking that ideas are novel, but most ideas from newcomers have likely been thought of before
- Coding is most of the work, but understanding, evaluating and integrating a suggesting change takes far longer. (Choice quote here: “Being given code is like being getting a puppy as a gift.”)
- Answering is as easy as asking, but a question that takes 5 seconds to ask can take 2 pages of careful writing to answer.
Finally, stewards of OSS should always choose their obligations. Instead of trying to be constantly available/accountable, set working hours, choose the methods by which you interact with the community, control your workflow, etc. Additionally, attempt to rephrase requests to make actors and obligations explicit, as well as using measurable verbs. Those things will help to remove ambiguity and ensure all sides have the same understanding of the discussion.
QED: Question Educated Development
By Miki Rezentes
Miki started this talk by explaining how she used Q/A Flow when homeschooling her kids. By getting them used to repeatedly answering the same questions (“How many inches in a foot?”, “How many feet in a yard?”), so that when they were problem solving on a test, they’d run through the same questions in their head.
As developers, we are continually asking ourselves questions, but many senior devs have internalized those questions to the point where they won’t recognize that they’re asking themselves questions. When training juniors, though, those questions should be made explicit, and the best way to do that is to always make them explicit.
Miki then ran through a number of questions she’d received through a survey she ran, grouped into different categories. She ran through the questions pretty quickly and I wasn’t able to write most of them down. The categories, though, were:
- Product Questions, e.g. What are the bare essentials for this feature? (useful to ask continually, instead of just at the beginning)
- Team Questions, e.g. How can I explain this to a junior? or Should someone else be doing this?
- User Questions
- Intuition Questions
Interestingly, none of the responses to the survey would be considered security questions. Questions Miki suggested here were things like “Am I opening an attack vector?”, “Am I protecting the user?” and “What permissions does this feature require?”
Thankfully, Miki has already posted her slides which include the full set of questions she presented.
By Chris McCord
Chris’ talk hit a lot of the same points as his ElixirConf talk, so I wound up taking far fewer notes than I otherwise would have.
The state of frontend development is understandably complex. Moving logic to the client makes for better UX but can require duplication of effort (input validation, localization of data, etc must be done on the client in JS and on the server in Elixir.) A better approach (for non-UI-intensive, non-SPA apps), is to push data up to server, perform whatever computations need to happen there and then send the data back to the client to be displayed.
Chris tried this a few years ago in Ruby, but the end result was a very complicated, fragile system. The advent of WebSockets has simplified two way communication and the power of OTP vastly simplifies the process of keeping state on the server.
A smattering of things that are new since ElixirConf (or I didn’t pick up on earlier):
- The server currently pushes down a lot of HTML and lets the client figure out how to change the DOM, but eventually LiveView will do DOM diffing itself and send far less data down the wire.
- Components are resilient and can recover from crashes, reverting back to a known good state, a la Scenic UI.
- To that end, state can be stored on the client for recovery
- No one has access to the repo yet (not even Jose) but LiveView will be released and versioned separately from Phoenix.
One interesting anecdote, Chris applied to work at 37Signals but was rejected. Had he gotten that job, he likely would not have moved to Elixir, so in a way DHH is the true creator of Phoenix!
That One, For Now
By Ben Marx
Ben delivered a very low level explanation of how processes work in Erlang. The design and sophistication of the BEAM is always impressive, but seeing snippets of the underlying C code is a good reminder that there’s no magic here. Just years of effort from a lot of people.
An Erlang process has four blocks of memory: a heap, a stack, a mailbox and a process control board. The heap and stack do normal heap and stack things (but are allocated together), the mailbox stores process to process messages and the PCB manages the state of the process.
There are 7 states for processes, but only 3 are generally relevant: Runnable, Running and Waiting. (A process’ state can be discovered programmatically, but you’d generally use the inspector for such a thing.) A waiting process has no messages in its mailbox and nothing to do. A runnable process has either received a message, hit a timeout or used up all of its reductions (more on reductions later). A running process is doing just that, executing code.
By default, Erlang allocates a scheduler per CPU core. That value can be overridden, as can the rules for how schedulers are bound to cores, etc, but it’s generally best to trust the BEAM. Schedulers are preemptive, meaning they will interrupt long running processes. Instead of preempting based on the clock, reductions are used to measure computation length. I’m not clear on exactly what constitutes a reduction, but I think they’re somehow tied to function calls. By default, all processes are preempted after consuming 4k reductions.
Each scheduler is assigned a dedicated run queue, which is just a list of waiting and runnable processes. (Having a queue per scheduler reduces lock contention as schedulers look for processes to run.) Processes can be assigned priorities, and lower priority processes are forced to wait for multiple passes through the run queue before being scheduled, so that higher priority processes get scheduled multiple times.
There is also a load balancer that is responsible for keeping queues of a reasonably similar size. The goal is to give work to the fewest number of schedulers without overloading the CPU. The load balancer is capable of stealing work from other schedulers when idle.
Again, the capabilities of the BEAM are amazing and I’m reminded of a point that Chris Keathley made yesterday. So many of us are here because of the platform, because of the BEAM and what it allows us to build.
Crypto + Concurrency
By Anna Neyzberg
Anna’s talk was an interesting blend of introduction to cryptocurrency/blockchain and introduction to OTP. The first section of the talk was explaining the algorithm of combining transactions into blocks, building Merkle trees and then mining the block by discovering a nonce that results in a hash starting with a certain number of zeros.
We can mimic that process with the building blocks of Elixir and OTP:
- A Transaction module for signing and verifying transactions
- A Block module for combining transactions
- A BlockChain module for combining blocks
- The chain is stored using an Agent and supervised to recover from failure
- Modeling nodes with GenServers can mimic the gossip protocol used to distribute mined blocks
- Distributed Erlang lets you run nodes on separate machines.
By Dave Thomas
The final talk of the day was a very optimistic one from Dave Thomas. Dave began by saying that he is trying to get away from negative statements like “Agile is dead” and instead would like to argue that both OO and FP are alive and well, we’re just doing them wrong.
After an overview of the early OO languages (Simula and Smalltalk) and a critique of C++ (which he labeled a “Class Oriented Language”), Dave presented his Tao of OO:
Work performed by small, independent, opaque processing units that communicate by message passing.
Dave then gave an overview of FP principles (continually apologizing to John Hughes along the way), focusing on immutability and first class functions. Reframing immutability from the requirement that values don’t change to the design principle that changed values are never shared will prove to be helpful shortly. Dave’s Tao of FP is stated as:
Immutable data transformed by pure functions in the presence of algebraic types and lazy evaluation
Dave sees three main areas of overlap between OO and FP:
- Non shared state
- Transformed by functions
- Invoked by message passing
Elixir is not object oriented and is not functional. Joe Armstrong calls it concurrency-oriented. But why do we need a label at all? It is simply a language and an environment. And we can steal the best ideas from both paradigms.
To begin with, there are 2 models of processing:
- Stateless, pure functions that transform input to output.
- Stateful functions where part of the output is fed back to the next invocation as input, forming a feedback loop. This is very natural to Elixir developers already. Crucially, the state here needs to be obvious, and local. Callers should not be able to introspect it or extract data from it without calling a function.
There are four levels or groupings of things:
- Functions, which only apply to model 1.
- Modules, which are stateless libraries when composed of pure functions.
- Components, something like a business logic library.
- Deliverables, things that are actually shipped.
Both components and deliverables would Elixir applications, created by
mix new but are very different things.
Good ideas that should be used in Elixir:
- Encapsulation / Hide the implementation. This can be accomplished by keeping the state opaque. Dave is also a fan of breaking up the implementation of GenServers, described in detail here.
- Single Responsibility Principle. Processes are cheap and easy, use them liberally. Describe what you’re building in a single sentence. Whenever the word ‘and’ appears, break the thing in two and repeat until there are no ‘and’s left.
- State Invariants
- Create a process with a valid state
- Assume state is valid when entering process
- Ensure state is valid when exiting
- Encapsulate variability
- Isolate things that change into their own chunks of code
- When similar things are done in multiple places, extract the generalities and pass in functions that encapsulate the differences (e.g. iterating over a data structure and executing a callback function for each element)
The overarching points Dave made were:
- Maximize cohesion
- Minimize coupling
- Make it easy to change, because “well designed code is easy to change”