Tools for writers

Update 2021-04-26: I wanted to link readers to this fantastic post at Programming Historian, “Sustainable Authorship in Plain Text using Pandoc and Markdown”, by Dennis Tenen and Grant Wythoff. It's aimed at an academic audience but it makes some of the same points I do below, but more elegantly. I had intended to link in the original post but did not!

Why I got mad today!

This morning the YouTube algorithm led me from something about old computers to a video by a writing influencer outlining the “top ten tools for writers 2020” or something along those lines. (I'm not here to pick fights, so I won't name names.)

I left it running and I was disappointed to see that the recommendations veered between the pedestrian and the pernicious.

There were some things that were harmless enough. One thing that was perhaps relevant to fiction writers was a character name generator.

The recommendations for actual writing tools were the weirdest, and ever so slightly got my goat.

Those recommendations were fo Microsoft Word, Google Docs, and Scrivener, with something called Hemingway thrown in as an editing tool.

I found it puzzling that first two – which are ubiquitous – were considered as standing in need of promotion.

All software has fans, and I am not here to talk you out of using Word if you like it, or if you have to use it for whatever reason, for example as a condition of employment. But I'll briefly say why I think each of these are bad for writers.

Microsoft Word is simply not fit for purpose for most writers. It is neither a great text editor nor a good desktop publishing package, despite trying to offer functions in both of these areas.

It saves documents in a format which is difficult to open in other software.[1] It is not very user friendly, because the company who makes it is more incentivized by selling “good enough” bundled office solutions to large enterprises than it is by what end users might want to do.

It also, one way or another, costs you money. You will need a license, and as time goes on, you will find that newer versions of Word and/or its parent operating system will not run on otherwise perfectly functional hardware.

Google Docs — a dumbed-down Word knock-off in the cloud — is notionally free. But as usual with that company, the product is you.

To use it, you must consign your work to Google's vast, data-devouring maw, which it will then align with everything else it knows about you. You must trust the company's limited assurances concerning privacy and security. And you may fall into the bad habit of trusting that Google's cloud (i.e. its vast arrays of warehoused computers) will never suffer any downtime. [2]

Scrivener is proprietary software, which uses an idiosyncratic file format.

Nevertheless, it has passionate fans – some friends of mine swear by it. Those who do, however, have used it to write books. It may perform well in that very specific endeavor. For shorter form writing on a faster cadence, it has a number of shortcomings.

Like MS Word, it only opens one file at a time, so one is forever going back and forth to other files one wishes to refer to. It has a lot of bells and whistles involving formatting text, something which, beyond some very basic gestures, I do not consider to be the job of the writer, but the editor and publisher, and is certainly not something that is often required of me.

It is also, I think, a complicated tool that seeks to impose a workflow involving skeumorphic index cards and corkboards. This all seems unnecessary to me. I tried it for a while about 8 years ago but I neither got the hang of it, nor saw the point. I tend to like things that get out of the way and let me do what I need to.

Hemingway appears to be a slightly gimcrack web application that promises to make your writing more readable. I rather think that the entire craft of writing is a process wherein you work on making your writing more compelling and readable, and I doubt that software developers can make an algorithm that does anything more than homogenize writing styles.

I got mad on Twitter about all this, but then I decided I should take it the next level – the blogosphere.

It says on my tax return that I am a journalist – so a kind of professional writer, I guess. If the Bureau of Labor Statistics is to be trusted, I even do a little better out of this than the average person in my job. Perhaps I should try to do a little influencing myself, particularly on others who are trying to do this work in as efficient and low-cost a manner as they can, in order to make it pay.

My view is that the whole business of writing has been made more complicated and expensive than it needs to be inthe interests of selling (or attracting data rich users to) unnecessary software.

My tools for writers

The following, then, is my list of top tools for writers.

1. Markdown

(Cost: Free)

If you haven't learned how to use the simple markup protocol known as Markdown yet, you simply must do so immediately.

As the Markdown Guide puts it:

Markdown is a lightweight markup language that you can use to add formatting elements to plaintext text documents. Created by John Gruber in 2004, Markdown is now one of the world’s most popular markup languages.

Gruber and Aaron Swartz worked together in designing a simple markup protocol as a way of making it easier for humans to write for the web – the idea was that it could be simply converted into HTML, which by contrast is awkward and ugly as a medium for writing.

But Markdown has become much more, and it's now possible to convert it (using tools discussed below) into every major file format, and almost any other one you may ever need.

Below is a screenshot of the passage above, as written right here on this blogging platform in Markdown.

Markdown in the wild

You may say that the links are a little ugly but that's because URLs are ugly, and it is so much better than writing them in HTML.

It has a number of enormous advantages.

There's more to say about how embracing plain text leads, eventually, to an extremely efficient workflow. I may write more on that at some point, for now check out the Plain Text Project for some resources and ideas.

My other recommendations below may conceivably change one day – Markdown is eternal.

2. Pandoc

(Cost: Free, though you can voluntarily support the project with donations)

Pandoc is, in technical terms, “a Haskell library and a standalone command-line program”. Haskell is a computer language, but that's not important right now. What is important is that, as its documentation says, “if you need to convert files from one markup format into another, pandoc is your swiss-army knife”.

Markdown is a markup format. So is HTML. The PDFs and Word documents are also renderings of markup formats – it's just that in those cases, we never generally see the underlying markup.

Pandoc – which is completely free and open source – converts from and to a very wide range of such formats. I use it for this purpose every working day. My workflow when writing stories is: – Make notes in Markdown – Write draft in Markdown – Convert into editor's preferred format and one or two others with Pandoc – File the story in the editor's preferred method.

Some editors want stuff filed to Google docs. Fine. I convert to docx and upload, and make a PDF copy for me (that's just what I prefer to read copy in, since I have an e-reader).

Some editors want copy pasted into emails. Fine. I either convert to HTML, open in a browser, copy and paste into the email, or use a Thunderbird extension to render Markdown into html right there in the email.

The simplest and easiest way to see how this works is to download and use Panwriter, a free, open source Markdown editor, which works on Mac, Windows, and Linux, that allows you to convert Markdown into a number of formats right there in the application.

Pandoc's real power, though, is only revealed if you do something that many writers are reluctant to: acknowledge that they are using a computer, and open a terminal. By using the terminal's text only “command line”, and also templates written in another markup format called LaTeX, you can create beautifully typeset documents with practically limitless formatting options.

This, perhaps, is another rabbit hole for another time. But Pandoc's documentation is very good if you want to explore it yourself.

3. A text editor, any text editor, many text editors

(Cost: Various, mostly free with some offering extra goodies for $$$)

I have written in Markdown for many years – I can't remember when I stopped being word processor first but it must be around fifteen years ago. I did, however, stay on on way too long as a frustrated Mac OS user because I was in love with Ulysses, an admittedly very beautiful text editor which is often presented as Scrivener's competition.

Back then my mindset was that a person should have one perfect writing tool that they spent all their time in. When I figured out the obvious — that I could write text in any old editor — I realised that this wasn't true.

Admittedly, I now spend most of my time in Obsidian, which goes way beyond being a text editor – I would call it the first tool that reveals the true potential and power in a plain text workflow. More details can be for another day, but suffice to say that what Obsidian encourages is the construction of a whole system of knowledge and work in a single desktop directory (or folder in Windows-speak.

You can download Obsidian for free, without even signing up for an account. I choose to pay $4/month which gives me encrypted syncing across my working devices, and some assurance that the tool will continue to be developed. I am considering a more substantial contribution for the latter purpose.

But as a much happier Linux user, I have many other text editors installed which I use all the time, depending on how I am feeling. I just ensure that I save everything to the folder that serves as my Obsidian vault.

The text editors I have installed include:

All of these are either cross-platform (including mobile platforms) by design, or have exact functional equivalents in other operating systems.

Sometimes, for distraction free powerdrafting, I drag out my AlphaSmart Neo2, which I got on eBay a few years back for around $15. When I am done writing I hook it up to my computer via USB and watch it retype it all. Fun!

This all might seem like overkill, or to have the potential for distraction, but for me it works the opposite way. Once you have embraced the mindset that says that what you are producing in the first instance is plain text Markdown documents, it really doesn't matter too much how you get that done. You're not tied to any tool in particular, you just need to use the best one for the job, or the one that makes you feel best about doing it.

4. A filing system

(Cost: Free after some set-up time)

At one time, I really didn't have much of a digital filing system, and I relied on Spotlight searches to find things. I now consider this to literally have been the workings of a disordered mind (I had undiagnosed and untreated ADHD until the beginning of 2020).

You need to have a system that allows you to know where to locate anything without too much fuss. I now use, with great profit, a version of the Johnny Decimal System, which allows me to locate anything – my own writing, research documents, invoices, manuals, receipts, tax returns – with the greatest of ease.

5. A pocket-sized notebook and a pen in your top pocket (Cost: varies, at least $3 I figure)

You never know when you might need to jot something down.

Final thoughts

This whole system is platform and OS agnostic.

My operating system happens to be free and open source, but I could do it all just as easily on Mac or WIndows – all I would need is a few minutes to sync my work from Obsidian or Standard Notes and/or load it from physical backup.

I have, as already mentioned, an encrypted backup of all my work on a keychain. I have another flash drive with my operating system on it. I could go to the airport with no luggage, and as long as my destination has pawn shops, I could start up again within minutes.

I can often do things very quickly, without needing to switch between applications.

I can work on any machine I come across. The machines I myself have are used, old, and cheap, and everything works like a dream. Most of the software is both free (libre) and free (as in beer). I do offer monetary support to the things I most appreciate and would hate to disappear, but someone looking to start in this business on the cheap could defer the moment at which they start making a contribution to the projects they value.

I have control of all my data, and I know where everything lives.

Now, I'm a freelancer. This comes with many disadvantages, but one of its advantages is that I don't have a corporate IT department breathing down my neck, but us freelancers need to claim these advantages where we can.

So how much does all of this cost?

I think I need to deal with this in detail in a subsequent post as I don't wish to mislead, but very roughly – startup costs to sustain this workflow with two used computers; a new, large, and frivolous monitor; some data storage devices and a basic keyboard is way less than $2k, it's more like US$1500.

I am very comfortable amortizing this over at least 3 years – say $600/year to be on the safe side.

Ongoing costs including Internet/Phone, software subscriptions, donations/support to KDE and Pandoc... It's around $120/month (with some calculated as a share of household costs) and falling as I figure out that there are legacy costs which are no longer necessary.

That's $1440/yr.

All of that doesn't include the space I use as a proportion of my housing costs because, well, I am trying to figure that out right now for my accountant. It also doesn't include the extras (mostly subscriptions to various proprietary databases and the like) that make my life as a reporter easier – this is just the stuff that makes being a writer possible.

So roughly $2000/yr all up just to get to the launchpad. Not bad, and it's not like I am feeling deprived here. In truth, the technologies we use every day actually matured a long time ago. If you're folowing a workflow like this, even if you are using a modern browser and media players, you can do most things you need to on a five or possibly even a ten year old computer.

Like I said, in a subsequent post I will maybe square this off with the costs using the kinds of tools that are frequently recommended by influencers.

There's much more to say about this workflow, and this mindset. Soon one realises that almost everything is a text document at bottom (like CSVs and even SQL databases), and when reporters, in particular, realise that, it means you can process, say, document dumps with ruthless efficiency.

But more on all of that later, the important thing is that I got to vent about a YouTuber!

[1]: I regularly try to work with docx documents in the supposedly compatible LibreOffice suite, AMA. [2]: This recently happened to a friend who, during Google's recent downtime, lost access to the only copy of their manuscript.