making computers better

how we create and deliver software

Perhaps this is the part of the essay where you expect me to go on a rant about the merits of strong typing, the deficiencies of ORMs, or the inscrutability of Git commands.

But on the contrary, I think we're in a golden age for programming languages, library ecosystems, and development environments.

Consider:

• Beautiful and expressive syntaxes like Ruby, Python, Go, Rust, React, SQL, Swift, and HTML/CSS

• Code editors like vim, Sublime, Xcode, and VS Code

• Collaboration tools like GitHub, Slack/Discord, and Zoom/screenshare

• Self-learning resources ranging from Stack Overflow to YouTube educators and Twitch programming live streams

Where I see the potential for improvement in software creation is developer workflows.

Computing is at its best when it’s fun, fast, and ceremony-free. The user (or developer) can focus on what they want to get done with a minimum of steps and fussy errors. Compare a smartphone to a desktop computer in terms of those qualities. What's the equivalent of a smartphone-like experience for development?

Three workflow improvments I’ll talk about here:

1. Joining a project. If I invite another developer to edit my software, how fast and easy for them to run the project within a development environment and make their first change?

2. Seeing changes. How much time elapses between me making a code change and seeing the result of that change in its native context?

3. Publishing to users. Once I have a change working the way I want in my dev environment, how long does it take to publish that change to one or more of my users?

Taking each in turn…

Joining a project. By this I mean any time someone needs to set up a development environment to run and edit a given project. That includes me starting a new project in some language/framework; a new person I've invited to join my existing project; me editing my existing project on a new computer; or me returning to one of my old projects that I haven't worked on in months or years.

As software development stacks have grown in complexity, so has the difficulty of onboarding new people to a project. Companies with big codebases have entire teams dedicated to building out the onboarding experience. It may take days or weeks of effort to get a new developer up and running.

Even a smaller codebase may require lots of trial-and-error: Xcode projects that require confusing developer account key setup or front-end web projects with a maze-like stack of build tools. Setup instructions documented in a README are almost always out of date.

A leap forward in the last decade was the widespread adoption of dependency management such as NPM, Crate, and Bundler. Dependency managers try to install everything you need to run the project. And importantly, they give a clear error when the project will not be runnable—for example, if you have the wrong version of the language runtime.

Another technique is wrapping up projects into container images with tools like Vagrant or Docker. This is the right spirit, but its use is a bit heavyweight, so most projects don’t bother.

The future solution I’d like to see is a one-click, can’t-fail project installation. I want joining a new project to be as easy as clicking a link to a Google Doc where I can immediately start making edits. That probably requires a toolchain that is fully self-describing—like dependency managers, but extended to absolutely every aspect of setting up a project including environment configuration and external services.

One move in this direction is web-based all-in-one development environments such as Glitch, Repl.it, and Codespaces. So far none of these have become widely popular with developers, who typically prefer local and modular toolchains.

Seeing changes. Once upon a time, developers had an extensive compile-run cycle that could take minutes or longer. When I worked in the game industry doing Dreamcast and Playstation 2 game development, I had as much as a fifteen-minute wait on my build cycle. I kept a book by my desk and got a lot of reading done in those days.

PHP and other early web dev tools offered a save-reload loop that took just seconds—a revelation for me after years of game development. More recently, live-reload dev servers go a step better, removing the need to switch windows and press the reload button.

iOS and Android default dev kits still have a somewhat long cycle to get the program built and running on-device. Some newer frameworks like Flutter and SwiftUI can hot-reload changes.

All of this takes us gradually closer to the ideal state: perfect immediacy. I make a change, I see it; no waiting. We’re not there yet, but I hope we might achieve it in the coming decade.

Publishing to users. In ages past, we might share our software by putting the source code into a magazine or selling it in a box in a software store. More recently, you might send someone the .EXE via email or let them download it from your website.

Direct distribution of .EXEs was great in many ways, but opened the door for a profusion of malware turning computers into legions of zombie botnets. The solution has been a combination of permissions/sandboxing and app stores.

Curated app stores help with malware and discovery, which is a clear win. But the intermediation between a software creator and the user has many downsides. Permission to send bugfixes to your users, unknown review timelines, rejections for seemingly arbitrary reasons—it kills innovation, momentum, and simple joy of building and shipping good products to your users.

The iOS App Store is infamous for its heavy developer account setup (DUNS number, anyone?) and an opaque review process. And lest you think Apple is alone here, just check out every other vendor’s app store: the Chrome Web Store, the Windows store, Steam and all gaming consoles, Linux stores like Ubuntu and Elementary, and many others—all with their own variations on submission guidelines, review hurdles, and app permissions and sandboxing.

I can send a tweet that's published to potentially millions of users in seconds. Why can't I do the same with publishing my software?

Alongside this consolidation on the app store model, we have a quiet juggernaut: the web. The miracle of the web is that anyone can visit a website, download the software program described there, and execute it instantly and safely.

Only the web was built from the ground-up with fine-grained permissions and a language runtime with near-perfect sandboxing. There's a lot that native apps can do that the web can't, like high performance and tight hardware integration (and I don’t think the web will ever catch up). But even so, the simple power of the web-based delivery mechanism is so good that I expect it to continue to dominate software creation and delivery for the foreseeable future.

One swing I took at this problem space is Heroku. As great as the web is at delivering software from a server, you still need to do a bunch of time-consuming and error-prone installation and configuration ceremonies to set up and deploy to that server. Heroku takes that all away: with a single command you go from code on your computer to working software in the hands of your users. Netlify, Vercel, Digital Ocean’s App Platform, and others have continued this instant-deployment pattern, and as a software developer I can't imagine working any other way now.

Putting together these three workflow improvements (fast ways to join a project, see changes, and publish) and you get the the live coding experience seen in Codepen, JS Fiddle, and Observable. This is not quite direct manipulation for software creation, but it's a close as we've gotten so far, and I hope dev tool creators keep pushing it further.

And one more thing. If we can somehow reach a state where joining, seeing changes, and publishing are effortless and instantaneous, that opens up the final frontier: end-user programming.

End-user programming blurs all the lines: developer and user can easily be the same person; software creation can be a more collaborative and inclusive experience rather than siloed to professional software engineers; and every person in our society gets to experience (if even briefly) the satisfaction, empowerment, and analytical mindset that we programmers enjoy in our daily work.

The world we have: Programming tools are good, although still out of reach for most people. Project setup and seeing results of changes are slow and clunky. Software delivery is increasingly intermediated by app stores.

The world I want: Creating software feels like direct manipulation—as fast and fun as using a smartphone. Delivering software to users is instantaneous, like sending a tweet.