Been running Smarlist for about 6 years. Currently running the 'latest' version 3.15.
Upon examing my log files I have found that SmartList is not handling bounces at all. This has been going on since June 2000. I'm not sure what if anything I changed back then, perhaps upgraded sendmail? procbounce never gets executed and all the files in my bounces directory are dated June 2000.
At the bottom of this email is a test input msg. I am using the stock rc.request but the recipes are NOT matching on ANYTHING bounce related. In particular, for any bounced msg, I see from the log file:
procmail: Executing " sed -e $cutoff_bounce' q' >tmp.request" procmail: Assigning "LASTFOLDER= sed -e $cutoff_bounce' q' >tmp.request" procmail: No match on "^Subject: (Warning - delayed mail|(WARNING: message ([^ ]+ )?|Mail )delayed|(Returned mail: )?(warning: c(an|ould )not send m(essage fo|ail afte)r|Unbalanced '"'|Cannot send (within [0-9]|8-bit data to 7-bit)|Data format error|Headers too large|Eight bit data not allowed|Message (size )?exceeds (fixed )?maximum (fixed|message) size)|Undeliverable (RFC822 )?mail: temporarily unable to deliver|*** WARNING - Undelivered mail in mailqueue|Execution succee?ded)" procmail: No match on "^Subject: (Returned mail: )?(Service unavailable)" procmail: No match on "^Subject: (Warning from|mail warning| ?Waiting mail)" procmail: No match on "^(..?)?X-Loop: ()vulcan@vroc.org (bounce)" procmail: No match on "^Content-Type:[ ]*multipart/report;[ ]*/[^ ].*" procmail: Assigning "LOCKFILE" procmail: Unlocking "tmp.lock" procmail: No match on ! "< 524288" procmail: Match on "(^(Mailing-List:|Precedence:.*(junk|bulk|list)|To: Multiple recipients of |(((Resent-)?(From|Sender)|X-Envelope-From):|>?From )([^>]*[^(.%@a-z0-9])?(Post(ma?(st(e?r)?|n)|office)|(send)?Mail(er)?|daemon|m(mdf|ajordomo)|n?uucp|LIST(SERV|proc)|NETSERV|o(wner|ps)|r(e(quest|sponse)|oot)|b(ounce|bs.smtp)|echo|mirror|s(erv(ices?|er)|mtp(error)?|ystem)|A(dmin(istrator)?|MMGR|utoanswer))(([^).!:a-z0-9][-_a-z0-9]*)?[%@> ][^<)]*((.*).*)?)?$([^>]|$)))" procmail: Executing "formail,-A,X-Diagnostic: Mail coming from a daemon, ignored" procmail: Match on "^X-Loop: ()vulcan@vroc.org" procmail: Executing "formail,-A,X-Diagnostic: Possible loopback problem" procmail: No match on ! "^X-(Diagnostic|Processed):" procmail: Assigning "INCLUDERC=" procmail: No match on ! "^X-(Diagnostic|Processed):" procmail: Executing "formail,-AX-Envelope-To: vulcan-request" procmail: Match on ! "." procmail: Locking "request.lock" procmail: Assigning "LASTFOLDER=request" procmail: Opening "request" procmail: Acquiring kernel-lock procmail: Unlocking "request.lock"
Thus the email get dumped into the request folder.
Why in the world would the below msg not match on that? The receipe snippet from rc.request is
# # Enable special handling for DSNs #
:0 A * ^Content-Type:[ ]*multipart/report;[ ]*/[^ ].* * ^Mime-Version:.*1.*..*0 * MATCH ?? report-type="?delivery-status"? * B ?? ^Content-Type:.*message.*delivery-status { # If there were no fatal errors, drop it :0 B * ! ^Status:[ ]*5[ ]*. /dev/null
isadsn=yes }
# # Anything that still survived is most likely to be a bounce message. #
:0 Ahfw * ! ^X-Diagnostic: | procbounce
Also, has anyone written/updated the procbounce processing to handle the multiple recipients listed in the DSN msg? Looking at the FAQ and various other places, I see that was on a TODO list.
I've create a small test setup where I extracted the rc.request bounce related recipes and can try to debug this on the command line. Before I invest a lot of time in this, I want to know if anyone else has already done this.
HELP! Hope I have provided some information for us to work with.
From MAILER-DAEMON@vroc.org Sun Apr 8 00:15:26 2001
Received: from localhost (localhost) by nomad.vroc.org (8.11.0/8.11.0) id f384FPi19035; Sun, 8 Apr 2001 00:15:26 -0400 Date: Sun, 8 Apr 2001 00:15:26 -0400 From: Mail Delivery Subsystem MAILER-DAEMON@vroc.org Message-Id: 200104080415.f384FPi19035@nomad.vroc.org To: test-request@vroc.org MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary="f384FPi19035.986703326/nomad.vroc.org" Subject: Returned mail: see transcript for details Auto-Submitted: auto-generated (failure)
This is a MIME-encapsulated message
--f384FPi19035.986703326/nomad.vroc.org
The original message was received at Sun, 8 Apr 2001 00:15:25 -0400 from list@localhost
----- The following addresses had permanent fatal errors ----- kbass2_b@clark.net (reason: 550 Unknown local part kbass2_b in kbass2_b@clark.net) kbass_bounceme@clark.net (reason: 550 Unknown local part kbass_bounceme in kbass_bounceme@clark.net)
----- Transcript of session follows ----- ... while talking to mx2.veriomail.com.:
RCPT To:kbass_bounceme@clark.net
<<< 550 Unknown local part kbass_bounceme in kbass_bounceme@clark.net 550 5.1.1 kbass_bounceme@clark.net... User unknown
RCPT To:kbass2_b@clark.net
<<< 550 Unknown local part kbass2_b in kbass2_b@clark.net 550 5.1.1 kbass2_b@clark.net... User unknown
--f384FPi19035.986703326/nomad.vroc.org Content-Type: message/delivery-status
Reporting-MTA: dns; nomad.vroc.org Arrival-Date: Sun, 8 Apr 2001 00:15:25 -0400
Final-Recipient: RFC822; kbass2_b@clark.net Action: failed Status: 5.1.1 Remote-MTA: DNS; mx2.veriomail.com Diagnostic-Code: SMTP; 550 Unknown local part kbass2_b in kbass2_b@clark.net Last-Attempt-Date: Sun, 8 Apr 2001 00:15:26 -0400
Final-Recipient: RFC822; kbass_bounceme@clark.net Action: failed Status: 5.1.1 Remote-MTA: DNS; mx2.veriomail.com Diagnostic-Code: SMTP; 550 Unknown local part kbass_bounceme in kbass_bounceme@clark.net Last-Attempt-Date: Sun, 8 Apr 2001 00:15:26 -0400
--f384FPi19035.986703326/nomad.vroc.org Content-Type: text/rfc822-headers
Return-Path: test-request@vroc.org Received: (from list@localhost) by nomad.vroc.org (8.11.0/8.11.0) id f384FPh19035; Sun, 8 Apr 2001 00:15:25 -0400 Resent-Date: Sun, 8 Apr 2001 00:15:25 -0400 Date: Sun, 8 Apr 2001 00:15:18 -0400 (EDT) From: Ken Bass kbass@clark.net X-Sender: kbass@kbass To: test@vroc.org Subject: eee Message-ID: Pine.LNX.4.04.10104080015140.2682-100000@kbass MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Resent-Message-ID: wBK2iB.A.UpE.dX-z6@www.vroc.org Resent-From: test@vroc.org X-Mailing-List: test@vroc.org archive/latest/32 X-Loop: test@vroc.org List-Help: mailto:test-request@vroc.org?subject=help List-Subscribe: mailto:test-request@vroc.org?subject=subscribe List-Unsubscribe: mailto:test-request@vroc.org?subject=unsubscribe List-Post: mailto:test@vroc.org Precedence: list Resent-Sender: test-request@vroc.org
--f384FPi19035.986703326/nomad.vroc.org--
I found that the '# Enable special handling for DSNs' recipe has a problem. The [] classes should contain a space and a tab, but in the rc.reqest I have, as well as the one I just downloaded from the distribution on procmail.org, only a tab is present. No match can ever occur.
Is vi and/or emacs messing this up when I load the file? Like I said, even the one I downloaded from procmail.org diffed identically to my file before I added the spaces and fixed the recipe.
On Sun, 8 Apr 2001 12:25:39 -0400 (EDT), Ken Bass kbass@clark.net wrote:
Been running Smarlist for about 6 years. Currently running the 'latest' version 3.15.
Upon examing my log files I have found that SmartList is not handling bounces at all. This has been going on since June 2000. I'm not sure what if anything I changed back then, perhaps upgraded sendmail? procbounce never gets executed and all the files in my bounces directory are dated June 2000.
At the bottom of this email is a test input msg. I am using the stock rc.request but the recipes are NOT matching on ANYTHING bounce related. In particular, for any bounced msg, I see from the log file:
procmail: Executing " sed -e $cutoff_bounce' q' >tmp.request" procmail: Assigning "LASTFOLDER= sed -e $cutoff_bounce' q' >tmp.request" procmail: No match on "^Subject: (Warning - delayed mail|(WARNING: message ([^ ]+ )?|Mail )delayed|(Returned mail: )?(warning: c(an|ould )not send m(essage fo|ail afte)r|Unbalanced '"'|Cannot send (within [0-9]|8-bit data to 7-bit)|Data format error|Headers too large|Eight bit data not allowed|Message (size )?exceeds (fixed )?maximum (fixed|message) size)|Undeliverable (RFC822 )?mail: temporarily unable to deliver|*** WARNING - Undelivered mail in mailqueue|Execution succee?ded)" procmail: No match on "^Subject: (Returned mail: )?(Service unavailable)" procmail: No match on "^Subject: (Warning from|mail warning| ?Waiting mail)" procmail: No match on "^(..?)?X-Loop: ()vulcan@vroc.org (bounce)" procmail: No match on "^Content-Type:[ ]*multipart/report;[ ]*/[^ ].*" procmail: Assigning "LOCKFILE" procmail: Unlocking "tmp.lock" procmail: No match on ! "< 524288" procmail: Match on "(^(Mailing-List:|Precedence:.*(junk|bulk|list)|To: Multiple recipients of |(((Resent-)?(From|Sender)|X-Envelope-From):|>?From )([^>]*[^(.%@a-z0-9])?(Post(ma?(st(e?r)?|n)|office)|(send)?Mail(er)?|daemon|m(mdf|ajordomo)|n?uucp|LIST(SERV|proc)|NETSERV|o(wner|ps)|r(e(quest|sponse)|oot)|b(ounce|bs.smtp)|echo|mirror|s(erv(ices?|er)|mtp(error)?|ystem)|A(dmin(istrator)?|MMGR|utoanswer))(([^).!:a-z0-9][-_a-z0-9]*)?[%@> ][^<)]*((.*).*)?)?$([^>]|$)))" procmail: Executing "formail,-A,X-Diagnostic: Mail coming from a daemon, ignored" procmail: Match on "^X-Loop: ()vulcan@vroc.org" procmail: Executing "formail,-A,X-Diagnostic: Possible loopback problem" procmail: No match on ! "^X-(Diagnostic|Processed):" procmail: Assigning "INCLUDERC=" procmail: No match on ! "^X-(Diagnostic|Processed):" procmail: Executing "formail,-AX-Envelope-To: vulcan-request" procmail: Match on ! "." procmail: Locking "request.lock" procmail: Assigning "LASTFOLDER=request" procmail: Opening "request" procmail: Acquiring kernel-lock procmail: Unlocking "request.lock"
Thus the email get dumped into the request folder.
Why in the world would the below msg not match on that? The receipe snippet from rc.request is
# # Enable special handling for DSNs #
:0 A
- ^Content-Type:[ ]*multipart/report;[ ]*/[^ ].*
- ^Mime-Version:.*1.*..*0
- MATCH ?? report-type="?delivery-status"?
- B ?? ^Content-Type:.*message.*delivery-status
{ # If there were no fatal errors, drop it :0 B * ! ^Status:[ ]*5[ ]*. /dev/null
isadsn=yes}
# # Anything that still survived is most likely to be a bounce message. #
:0 Ahfw
- ! ^X-Diagnostic:
| procbounce
Also, has anyone written/updated the procbounce processing to handle the multiple recipients listed in the DSN msg? Looking at the FAQ and various other places, I see that was on a TODO list.
I've create a small test setup where I extracted the rc.request bounce related recipes and can try to debug this on the command line. Before I invest a lot of time in this, I want to know if anyone else has already done this.
HELP! Hope I have provided some information for us to work with.
From MAILER-DAEMON@vroc.org Sun Apr 8 00:15:26 2001 Received: from localhost (localhost) by nomad.vroc.org (8.11.0/8.11.0) id f384FPi19035; Sun, 8 Apr 2001 00:15:26 -0400 Date: Sun, 8 Apr 2001 00:15:26 -0400 From: Mail Delivery Subsystem MAILER-DAEMON@vroc.org Message-Id: 200104080415.f384FPi19035@nomad.vroc.org To: test-request@vroc.org MIME-Version: 1.0 Content-Type: multipart/report; report-type=delivery-status; boundary="f384FPi19035.986703326/nomad.vroc.org" Subject: Returned mail: see transcript for details Auto-Submitted: auto-generated (failure)
This is a MIME-encapsulated message
--f384FPi19035.986703326/nomad.vroc.org
The original message was received at Sun, 8 Apr 2001 00:15:25 -0400 from list@localhost
----- The following addresses had permanent fatal errors ----- kbass2_b@clark.net (reason: 550 Unknown local part kbass2_b in kbass2_b@clark.net) kbass_bounceme@clark.net (reason: 550 Unknown local part kbass_bounceme in kbass_bounceme@clark.net)
----- Transcript of session follows ----- ... while talking to mx2.veriomail.com.:
RCPT To:kbass_bounceme@clark.net
<<< 550 Unknown local part kbass_bounceme in kbass_bounceme@clark.net 550 5.1.1 kbass_bounceme@clark.net... User unknown
RCPT To:kbass2_b@clark.net
<<< 550 Unknown local part kbass2_b in kbass2_b@clark.net 550 5.1.1 kbass2_b@clark.net... User unknown
--f384FPi19035.986703326/nomad.vroc.org Content-Type: message/delivery-status
Reporting-MTA: dns; nomad.vroc.org Arrival-Date: Sun, 8 Apr 2001 00:15:25 -0400
Final-Recipient: RFC822; kbass2_b@clark.net Action: failed Status: 5.1.1 Remote-MTA: DNS; mx2.veriomail.com Diagnostic-Code: SMTP; 550 Unknown local part kbass2_b in kbass2_b@clark.net Last-Attempt-Date: Sun, 8 Apr 2001 00:15:26 -0400
Final-Recipient: RFC822; kbass_bounceme@clark.net Action: failed Status: 5.1.1 Remote-MTA: DNS; mx2.veriomail.com Diagnostic-Code: SMTP; 550 Unknown local part kbass_bounceme in kbass_bounceme@clark.net Last-Attempt-Date: Sun, 8 Apr 2001 00:15:26 -0400
--f384FPi19035.986703326/nomad.vroc.org Content-Type: text/rfc822-headers
Return-Path: test-request@vroc.org Received: (from list@localhost) by nomad.vroc.org (8.11.0/8.11.0) id f384FPh19035; Sun, 8 Apr 2001 00:15:25 -0400 Resent-Date: Sun, 8 Apr 2001 00:15:25 -0400 Date: Sun, 8 Apr 2001 00:15:18 -0400 (EDT) From: Ken Bass kbass@clark.net X-Sender: kbass@kbass To: test@vroc.org Subject: eee Message-ID: Pine.LNX.4.04.10104080015140.2682-100000@kbass MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Resent-Message-ID: wBK2iB.A.UpE.dX-z6@www.vroc.org Resent-From: test@vroc.org X-Mailing-List: test@vroc.org archive/latest/32 X-Loop: test@vroc.org List-Help: mailto:test-request@vroc.org?subject=help List-Subscribe: mailto:test-request@vroc.org?subject=subscribe List-Unsubscribe: mailto:test-request@vroc.org?subject=unsubscribe List-Post: mailto:test@vroc.org Precedence: list Resent-Sender: test-request@vroc.org
--f384FPi19035.986703326/nomad.vroc.org--
Smartlist mailing list Smartlist@lists.RWTH-Aachen.DE http://MailMan.RWTH-Aachen.DE/mailman/listinfo/smartlist
I have a one large list running alongside several others that has ceased to synchronize the dist and accept files.... Is there anything I can check to see what is happening?
all the best David
be sure that dist and accept are hard linked together.
Yoy can check by running ~list/.bin/showlink dist accept
If they arent, removed accept and do link dist accept
On Wed, 11 Apr 2001 14:08:14 +0200, "David Kelly" dkelly@ono.com wrote:
I have a one large list running alongside several others that has ceased to synchronize the dist and accept files.... Is there anything I can check to see what is happening?
all the best David
Smartlist mailing list Smartlist@lists.RWTH-Aachen.DE http://MailMan.RWTH-Aachen.DE/mailman/listinfo/smartlist
At 8:08 AM -0400 4/11/01, David Kelly is rumored to have typed:
I have a one large list running alongside several others that has ceased to synchronize the dist and accept files.... Is there anything I can check to see what is happening?
One can assume you broke the link between the two, probably by editing one of the files with some web-based editor that deletes and saves a file, or by using FTP to transfer one of the files in-and-out. I am assuming the dist file is the one that is being correctly updated, where the accept file is the one that "doesn't change" when a subscriber joins - to fix the problem, telnet into the machine, change directories to your list directory, and type at the prompt:
touch rc.lock; rm accept; ln dist accept; rm rc.lock
If things aren't where they should be, you can't access your list directory, you don't know what I mean by telnet, or you are dependant on some bizarre web-based control panel to operate your list, you will need to contact tech support for your hosting service. (Nothing personal, I just don't know whether you're running this on a machine you control, or a shared virtual machine.)
Charlie
At 11:58 -0400 11 Apr 2001, Charlie Summers charlie@lofcom.com wrote:
touch rc.lock; rm accept; ln dist accept; rm rc.lock
Using touch(1) to create the lockfile isn't a good idea. If the lockfile doesn't currently exist it will be fine, but if it's already there (and it could be created between the time ls is run and the time touch is run) touch won't complain at all.
Instead you should use the lockfile command (part of the procmail package, so it will exist on any system using SmartList):
lockfile rc.lock && rm accept && ln dist accept && rm rc.lock
At 1:37 AM -0400 4/12/01, Aaron Schrab is rumored to have typed:
Using touch(1) to create the lockfile isn't a good idea.
For the short time this ball is going to be in play, I don't see a major problem using touch over lockfile (indeed, for this, no lock file is really required, since AFAIK nothing in SmartList works directly with accept anyway).
Charlie
At 09:15 -0400 12 Apr 2001, Charlie Summers charlie@lofcom.com wrote:
For the short time this ball is going to be in play, I don't see a major problem using touch over lockfile (indeed, for this, no lock file is really
It doesn't take long for a race condition to cause problems. And since it's "lockfile" is only three characters longer than "touch" I don't see any advantage to using touch. Best to get in that habit anyway.
required, since AFAIK nothing in SmartList works directly with accept anyway).
Well, nothing modifies it. But It not being there could cause a message to be incorrectly rejected.
At 09:15 -0400 12 Apr 2001, Charlie Summers charlie@lofcom.com wrote:
problem using touch over lockfile (indeed, for this, no lock file is really required, since AFAIK nothing in SmartList works directly with accept anyway).
Forgot to mention this in my previous message. Since touch will succeed whether or not the lockfile had already existed, it may have been created by some other action. The lockfile created by that other command may then be removed before the action is finished, possibly allowing multiple simultaneous commands that do make modifications to the list.
On Thu, Apr 12, 2001 at 12:37:34AM -0500, Aaron Schrab wrote:
At 11:58 -0400 11 Apr 2001, Charlie Summers charlie@lofcom.com wrote:
touch rc.lock; rm accept; ln dist accept; rm rc.lock
Using touch(1) to create the lockfile isn't a good idea. If the lockfile doesn't currently exist it will be fine, but if it's already there (and it could be created between the time ls is run and the time touch is run) touch won't complain at all.
Instead you should use the lockfile command (part of the procmail package, so it will exist on any system using SmartList):
lockfile rc.lock && rm accept && ln dist accept && rm rc.lock
Or side-step the whole issue with an atomic operation:
ln -f dist accept
I mean, sheesh.
Charlie ... your answer appears to have worked (via ssh to a virtual host)...there was no rc.lock file in existance
the lockfile option created a prompt asking if I wanted to delete the accept file which I wasnt expecting...so I didnt do it!!!
thanks to everyone for their help David
----- Original Message ----- From: Charlie Summers charlie@lofcom.com To: David Kelly dkelly@ono.com; smartlist@lists.RWTH-Aachen.DE Sent: Wednesday, April 11, 2001 5:58 PM Subject: Re:dist and accept not synchronized anymore
At 8:08 AM -0400 4/11/01, David Kelly is rumored to have typed:
I have a one large list running alongside several others that
has
ceased to synchronize the dist and accept files.... Is there anything I can check to see what is happening?
One can assume you broke the link between the two, probably by
editing one
of the files with some web-based editor that deletes and saves a
file, or by
using FTP to transfer one of the files in-and-out. I am assuming
the dist
file is the one that is being correctly updated, where the accept
file is the
one that "doesn't change" when a subscriber joins - to fix the
problem,
telnet into the machine, change directories to your list
directory, and type
at the prompt:
touch rc.lock; rm accept; ln dist accept; rm rc.lock
If things aren't where they should be, you can't access your
list
directory, you don't know what I mean by telnet, or you are
dependant on some
bizarre web-based control panel to operate your list, you will
need to
contact tech support for your hosting service. (Nothing personal,
I just
don't know whether you're running this on a machine you control,
or a shared
virtual machine.)
Charlie
Smartlist mailing list Smartlist@lists.RWTH-Aachen.DE http://MailMan.RWTH-Aachen.DE/mailman/listinfo/smartlist
At 19:47 +0200 12 Apr 2001, David Kelly dkelly@ono.com wrote:
virtual host)...there was no rc.lock file in existance
There won't be most of the time. It should only exist when some action needs exclusive access to the list related files.
the lockfile option created a prompt asking if I wanted to delete the accept file which I wasnt expecting...so I didnt do it!!!
Both the version that I posted using lockfile and the version using touch contained a command to remove the accept file. It's necessary to do that before the name accept can be used as a link to the dist file.
smartlist@lists.rwth-aachen.de
-
Aaron Schrab -
Charlie Summers -
David Kelly -
Ken Bass -
Tim Pierce