Our Tools Suck

[marcmagus]

I just requested of a friend that they reconfigure their email client to send email with a text width <=78 characters, so that when I quote relevant portions of their emails they can have every line indented and prepended with ">" without the formatting getting all screwed up. How 1990s is that? Why haven't we come up with a decent way of indicating quotes which is readable for people using plain-text, easily formatted to be pretty looking in serious mail clients, and doesn't depend on the terminal width being 80 columns? For that matter, why do we still send email with hard breaks somewhere in the 70th-78th column, rather than only using hard breaks for paragraph separation and allowing the reader's client to word-wrap nicely for readability based on the size of their actual view? Why did GMail, last I checked, fuck it up so that in their default configuration, if you do internal quoting rather than top-quoting or bottom-quoting, the quoted text is completely indistinguishable from the content of the new email?

Our tools suck. Here's what *should* happen:

* Our mailers should smoothly word-wrap the text of incoming emails when "lines" are longer than the width of whatever they're using to display the text. The cost in processing power is really negligible in today's world (and is being incurred at the sending end now anyway).

* We should all be able to assume this is going to happen, and thus send email which only uses whatever newline character is being used when it's necessary to denote something meaningful.

* Our mailers should do something to distinguish quoted text from the rest of the email body (every mail client I've used in the last 5 years has done this one way or the other).

* We should be able to assume this is going to happen, and thus be able to come up with a nice, simple, universal new standard for quoting text which degrades reasonably for crappy clients but doesn't make replying using internal quotation difficult/annoying. Actually, if we stop using hard breaks at the sending end in favor of word-wrap at the receiving end, the current standard of prepending "> " should suffice, since we no longer have to worry that adding characters to a line will screw up its formatting in the average case.

Have I just misconfigured mutt and vim and consequently missed out on the last 10 years of development? I'll have to look into that; it wouldn't be the first time it had happened. However, we *are* still including hard breaks in our email, and stripping out someone's hard breaks is dangerous (they might actually indicate formatting).

Along the same lines, in writing this post I need to type in crap like "&lt;". What's with that shit? The HTML team made a really crappy choice when deciding which characters should have special meanings and what we should do when we want to use them the normal way. Why can't I escape special characters with a backslash, so it's "\<" like in every fucking other thing I use, is trivial to type (just hit backslash before what you want), and is easy to remember (just hit backslash before what you want). Fuckers. I may have to figure out how the JLJ preprocessor works so I can extend it to do exactly that, because this crap is stupid.

(The formatting on this is probably going to be FUBAR because I can't be assed to enclose every paragraph in <P> tags like I usually do, and I haven't yet learned how to use this tool if I'm not going to send true HTML. I should do that sometime; it's a nice tool in general.)

BTW, how do I get vim to show text as word-wrapped without inserting actual hard breaks in some form of text mode? I don't need a newline every 78 characters when I'm writing LJ posts, but it's much easier to read what I've written if the visual lines wrap at word boundaries instead of at the edge of the terminal, and if up/down go by the visual line rather than the real line. I'm sure someone's written something to do this and I just haven't found it yet.

Please don't tell me to switch away from the terminal. The terminal is exceedingly useful in a lot of ways, and by now we should have evolved our tools to take advantage of its advantages without putting us hostage to its apparent limitations (apparent because they're only limitations if we're lazy programmers).

Comments

[tymothytoastman.livejournal.com]

Simple CSS would do the trick, embed the original email in a div and just start nesting. *Sigh*

[marcmagus]

*puke*

Something like that could work, except that HTML/XML/etc. markup (specially XML) fails its primary mission of being unobtrusive in this situation.

Adopting <q> as the "standard" for marking quotes might just barely pass the criterion that it should be relatively unobtrusive and readable if it's being viewed in plaintext for some reason. Although using "HTML" in your text/plain part is a little ugly.

VIM Reformats Reply-Quoted Text

[dhs.livejournal.com]

I just requested of a friend that they reconfigure their email client to send email with a text width <=78 characters, so that when I quote relevant portions of their emails they can have every line indented and prepended with ">" without the formatting getting all screwed up.

You do know that after you have isolated the text you are responding to (because leaving in all the text you are not responding to for the recipients to wade through and try to figure out the context of your comment is rude (almost as rude as not leaving in any of what you are replying to and assuming that your readers will remember the thread in its entirety and figure out which part of if you are talking about (but I digress))) you can tell vim to reformat the paragraph you are on ('gqip') and it will Do The Right Thing(TM) with the reply markers? Having this facility means that you don't need to bother your correspondents to keep to some particular formatting, because as you need to respond, you can rewrap the text as needed.

Re: VIM Reformats Reply-Quoted Text

[marcmagus]

Yes, but I have to do it once for every bit of text I'm replying to, which is an unreasonable amount of effort (gqG messes up formatting a bit if there are nested quotes, IIRC). Keep in mind that I intersperse my text with the text I'm quoting. (I also recognize that I probably err on the side of leaving in too much of the original text, which is partly to do with my conversational style.)

I should be able to trim their text without having to mess with its formatting at all. Secondarily, either vim or mutt should Do The Right Thing before I start typing, but I haven't yet come up with a nontrivial way to accomplish that.

[mneme]

There -is- a standard here that's compatable with the pre-existing approach (i.e. doesn't break existing tools, as your approach would/does) -- format=flowed. In format=flowed, "cosmetic" 78 char newlines internal to a paragraph are prepended with a space. Real newlines following a paragraph.

The sidekick (which can't really use 78 char lines internally due to, you know, having a teeny tiny screen) uses this approach and it works great (although when I switched from using a palm to write zines to using a sidekick, I missed this crucial formatting nicety and ended up using a totally unnecessary "one newline=same para", "two newline = new para", "three newline = new section" approach instead; if I'd noticed I'd probably have just stripped " \n" before formatting and gotten to use the formatting protocol I used on the palm).

For that matter, as long as your mailer uses it, it can reformat to its hearts content.

The equivalent of \char in html is &#UNICODE; -- which is computationally similar, but, alas, much harder for real people to do. (ie, &#60; instead of &lt;, which at least has the advantage of not special casing every bloody character!).

[marcmagus]

Thanks for the pointer to text/plain; format=flowed. I'll look into it. But how does it interact with quoting? (Mutt says it ignores $indent_string when $format_flowed is on.) It looks at first glance like it solves one problem (indicate to the recipient's MUA that the paragraph should be reflowed appropriately to its display), but not the other problem (have a nice way of indicating a block of quoted text which should be similarly reflowed by the MUA, but will look at least somewhat reasonable in a non-supporting MUA)?

Are there tools in use which are broken by long lines inside an email? Are they not broken?

My complaints about HTML entities are twofold:
1) Way too many things need to use entities rather than the raw character according to the standard. (Ampersand, left and right angle-bracket, single and double quote, that I'm aware of, are potentially problematic.)
2) Those symbols are punctuation marks which actually come up on a regular basis.
3) Whereas to learn \& you need to learn that \ escapes the following character, to learn entities you need to memorize either the entity name or (worse) Unicode codepoint for the character you want.

IOW, I don't want the equivalent of \nnn (specifying by codepoint), I want escaping...make a character non-special by preceding it with a backslash. Merely extending HTML with \&, \<, and \\ would probably suffice.

[mneme]

Re format=flowed: it's a good question; I mean, I'm happy with "non-word characters in the left-hand side are quoting characters, and should be treated as same when reflowing" but it's possible one could come up with rare exceptions when that's not true.

The problem with longline wrapped text is that the 822 standard really does specify 80 character lines, so lots of tools build around that and the following standards don't reflow (hell, most of mine don't). They aren't -broken- by long lines, but they don't deal with them gracefully, and for the formats they're designed for, they're certainly not broken.

format=flowed works with them because it hardbreaks at 80 (and it works with things that want to reflow as well, for obvious reasons); longlines is suboptimal.

Re entities; blame SGML, which is identical on this respect. Well, kinda -- doesn't CDATA have different rules? I'm not sure whether one can use it as a (cumbersome) escaping technique. I agree with you in terms of a language meant to be written by human beings; the problem is that SGML is only sorta meant to be so, and isn't an enclosure mechanism (where \\ and \char are natural) but an encoding mechanism. Mind, you -don't- actually need to escape all that list; you only need to escape & and <. The only time you need to escape quotes is inside a tag, as quotes only have a special meaning inside a tag (just as / and > only have special meanings once you've hit a <).