Multipart/alternative/html/encoded woes ... I have my lists set up to block (reject) posts with HTML, but even this isn't really solving a whole lot of my problems, because people (mostly AOLers) continue to send messages in HTML and then wonder why they're not going through. Even though my reject message clearly states what the problem is and directs them to pages that have instructions for turning off the HTML in just about every mail program, every frickin' time AOL upgrades, it causes a whole new set of problems because the new program isn't on any of the instruction lists yet. So I'm wondering... is there any way [or can anyone here come up with a way] to make Smartlist simply convert any and all HTML/encoded messages to plain text? There has got to be *some* way to make that happen, doesn't there? I have other friends running mailing lists using other software programs and they've been able to do this, so it seems like Smartlist should be able to as well. The past few months a lot of my list members have taken to saying, "Smartlist isn't very smart, is it?" every time the program has difficulties handling particular types of messages. I just don't want to believe that because I'm sort of attached to it, but I must admit I'm starting to become more and more frustrated as times goes on. I swear I have just about every script possible in my various local and submit files, and I dunno..... It gets to a point where, when your whole program seems to be held together and controlled more by the spit-and-glue of numerous fixes than the basic program itself, you start wondering if the program you're using is the most efficient choice. If I had a wish list of things I'd change about Smartlist, the first thing would be to have some way to set it to strip HTML from messages. I think that should be a basic part of the program, because HTML encoding is one of the biggest problems facing most list owners these days. The second thing I'd change is the way the program cannot seem to interpret short messages with no subject line (or short subject lines) as posts, but flags them as "undecipherable" and sends out the help file. This is really annoying, and it happens all the time. A person on my list had a birthday today, and when someone attempted to send a message with the subject "30!" and a short note in the body wishing them a happy birthday, Smartlist bumped it and sent the help file. Unfortunately, I don't have the skills to create a script to fix either thing myself. But I know several of you do, or know others who do, so I'm putting this out there in the hopes that someone here might be able to come up with something that could help. Violet xoxox
On Thu, Mar 06, 2003 at 01:35:33AM -0800, Violet wrote:
The second thing I'd change is the way the program cannot seem to interpret short messages with no subject line (or short subject lines) as posts, but flags them as "undecipherable" and sends out the help file. This is really annoying, and it happens all the time.
Yep. The check for administrative-request-lookalike messages in rc.submit is too aggressive: it catches almost every message that (1) is short enough (less than six lines and 640 characters) and (2) doesn't begin with English letter (in addition to things actually recognized as requests). To make things worse, rc.request tends to interpret various oddities as subscription requests, so a normal (short) message may be rejected with "already on the subscriber list", which is rather confusing. I haven't quite decided what to do about it, but one solution would be trashing the test in rc.submit for good and accept administrative requests only when sent to -request address. For a somewhat milder solution, look at this pattern in rc.submit: * -100^0 B ?? ^^([ ]|$)*\ ((((archives?:?($|[ ]+)|\ ((send|get)(me)?|gimme|retrieve|mail|ls|dir(ectory)?|\ list|show|search|[fe]?grep|find|maxfiles|version|help|info)\ ([ ].*)?$)([ ]|$)*)+\ ([^ a-z].*$(.*$(.*$(.*$(.*$)?)?)?)?)?^^|\ (help|info)[ ]*$|\ (add|join|leave|sign( [^ ]+ |-)?o(n|ff)|(un|de)?-?sub)\>)|\ ([^ a-z].*$(.*$(.*$(.*$(.*$)?)?)?)?)?^^|\ .*( (join|leave|add .* to|(delete|remove) .* from|\ (take|sign|get) .* off|(put|sign) .* on) .* [a-z-]*list|\ (un-?|sub?)scri(be|ption))\>|\ ^^) Remove the two lines looking for non-ascii beginnings, i.e., to this: * -100^0 B ?? ^^([ ]|$)*\ ((((archives?:?($|[ ]+)|\ ((send|get)(me)?|gimme|retrieve|mail|ls|dir(ectory)?|\ list|show|search|[fe]?grep|find|maxfiles|version|help|info)\ ([ ].*)?$)([ ]|$)*)+\ (help|info)[ ]*$|\ (add|join|leave|sign( [^ ]+ |-)?o(n|ff)|(un|de)?-?sub)\>)|\ .*( (join|leave|add .* to|(delete|remove) .* from|\ (take|sign|get) .* off|(put|sign) .* on) .* [a-z-]*list|\ (un-?|sub?)scri(be|ption))\>|\ ^^) That seems to take care most of those errors. I haven't tested it very thoroughly however - caveat emptor. -- Tapani Tarvainen
On [2003-Mar-06] Violet <violet@torithoughts.org> wrote:
So I'm wondering... is there any way [or can anyone here come up with a way] to make Smartlist simply convert any and all HTML/encoded messages to plain text? There has got to be *some* way to make that happen, doesn't there? I have other friends running mailing lists using other software programs and they've been able to do this, so it seems like Smartlist should be able to as well.
I use stripmime.pl as a simple "take the plaintext part and dump the html part" filter. I still do not accept html-only message on my lists and no subscribers I have (including those AOL'ers) are unable to have mail sent as, at least, multipart/alternative so a text version is available. I simply put :0 fw * ^Content-type: .*multipart/(alternative|mixed) * B ?? Content-type: text/plain | perl $STRIPMIME in rc.local.s00 and be done with it. (stripmime is small and self-contained perl scropt and is available from http://www.phred.org/~alex/stripmime.pl) Rich -- richard_ball@merck.com (I regret the presence of the legal disclaimer but I have no control over it) ------------------------------------------------------------------------------ Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it. ==============================================================================
At 4:35 AM -0500 3/6/03, Violet is rumored to have typed:
every frickin' time AOL upgrades, it causes a whole new set of problems because the new program isn't on any of the instruction lists yet.
Won't be, either; AOL 8.0 CAN NOT send plain text email, joining MSN Messinger and some others. As one of the world's foremost proponents of rejecting HTML email, I have officially given up.
So I'm wondering... is there any way [or can anyone here come up with a way] to make Smartlist simply convert any and all HTML/encoded messages to plain text?
To add to the options listed by others, I use demime.pl ( http://scifi.squawk.com/demime.html ) which I find about a aggressive in converting to plain text as anything else around. It'll happily strip tags from text/html messages as well as the easier "use-the-text/plain-section" that others use; it also _tells_ you what it's done, what attachment or sections have been removed, etc. I hacked the lists to not only run the mail through demime, but also add a note to the bottom that the server has altered the message. I then send a mail to the sender, asking them to attempt to send plain-text mail (understanding that some, AOL 8.x for example, can NOT), pointing them to a web page with more detailed information (which also suggests they find a real Internet provider). This seems to help get the Outlook-Express-on-default crowd to send text, while still allowing AOLamers to post. Charlie
(Pete, perhaps this info would be a useful addition to the FAQ?) Thanks to all who responded to my HTML-to-plain-text question. Very good suggestions. Here's what we decided to do... I have a friend who has a [majordomo-run] list on a server where they set up a converter for HTML messages. I asked him what they were using, and he said demime (same thing Charlie said he was using). In case it's of use to anyone else wrestling with this problem, what he wrote me about it was: we're using something called demime. it's a perl script which is run on every message to every list before majordomo even sees it. it can also be configured to remove advertising that yahoo and other mail systems place in message footers, though we haven't taken it that step yet. more information at http://scifi.squawk.com/demime.html. we're probably going to stop using it soon as we convert to majordomo2 since that has lots of built-in mime-handling capability. but in the meantime, demime does a nice job. there's a similar tool called...um...stripmime? i think that's it. i know a few people who use that too and are happy with it as well. In checking out both stripmime and demime, my server admin feels demime is the more useful of the two, so that's the one we're going to try. I would say that I'd keep everyone up on how it goes, but if Charlie's already using demime, then it obviously works with Smartlist so I'm sure we can get it to work for us without too much difficulty. Violet xoxox
At 6:03 AM -0500 3/8/03, Violet is rumored to have typed:
I would say that I'd keep everyone up on how it goes, but if Charlie's already using demime, then it obviously works with Smartlist so I'm sure we can get it to work for us without too much difficulty.
It works fine...but it IS an invocation of perl, so I would suggest you use the same "traps" you're currently using to reject to send the messages to demime...that way, messages that are already text/plain won't unnecessarily invoke a bloated instance of perl uselessly. (After all, you're using SmartList instead of majordomo because of the higher efficiency, right? ;) If you run into trouble, want example recipies, or just want me to opine, contact me off-list. ;) Charlie
Mandi! Violet In chel dí si favelave... V> Unfortunately, I don't have the skills to create a script to fix either V> thing myself. But I know several of you do, or know others who do, so I'm V> putting this out there in the hopes that someone here might be able to come V> up with something that could help. You can use a script like sanitizer (google for ``anomy sanitizer'') to remove/convert attachment caming to the list. There's a plently of tool to manage MIME encoded message, some year ago i've done a script in shell and metamail, but i think that tools like sanitizer are more general and configurable. -- Le camere deliberano lo stato di guerra e conferiscono al Governo i poteri necessari. (art. 78 Costituzione)
participants (5)
-
Charlie Summers -
Marco Gaiarin -
Richard G. Ball -
Tapani Tarvainen -
Violet