Developing a mailman module

[jogr]

As mentioned bevor I am interested in a mailman module for froxlor. So I started to adapt the old syscp mailman module for froxlor (0.9.12).

The base of operations is as followed:

- installing a fresh froxlor version

- downloading the module from the syscp forum (version mailman-0.1.3.tar.gz)

- copying the files to the froxlor folder

 

I have run the following sql command

CREATE TABLE modules_mailman (
 id int(11) unsigned NOT NULL auto_increment,
 `type` enum('list','update','new','delete','system','del_archive') NOT NULL default 'list',
 customerid int(11) unsigned NOT NULL default '0',
 domainid int(11) unsigned NOT NULL default '0',
 listid int(11) unsigned NOT NULL default '0',
 list_name tinytext NOT NULL,
 list_webhost tinytext NOT NULL,
 list_owner tinytext NOT NULL,
 list_mailhost tinytext NOT NULL,
 transport tinytext NOT NULL,
 list_password tinytext NOT NULL,
 PRIMARY KEY  (id)
) ENGINE=MyISAM;

 

to guarantee that the database is up to date (no missing tables).

 

 

I have created a file 99.mailman.php in lib/navigation

<?php
return array (
   	'admin' => array (
           	'mailman' => array (
                   	'label' => $lng['admin']['mailman'],
                   	'show_element' => true,
                   	'elements' => array (
                           	array (
                                   	'url' => 'modules_admin_mailman.php?page=lists',
                                   	'label' => $lng['menue']['mailinglists']['mailinglists'],
                           	),
                   	),
           	),
   	),
);
?>

 

After that I added the content of lng/modules/mailman/german.lng.php to lng/german.lng.php

 

Doing so I got my entry in main navigation and a link to the old admin mailinglist menu.

 

I've tryed to use that link, but I got a "blank" page, which means that there are no system and froxlor mailinglists (which is correct, there are none defined on my system yet) but I am not able to adept the pathes for mailman. phpmyadmin shows, that there were mailman entrys in panel_settings, but it seems but they are not displayed. Can you give me a hint which file / entrys I have to change?

[d00p]

well, as Froxlor changed the way settings are read, you also have to add an entry in the settings-array to get it from the database, take a look at any of the files in /actions/admin/settings/, copy one of the files and rename it to [freenumber].mailman.php then edit. I hope the array-keys are selfexplaining

 

But _then_ your settings are available as you described it yes

[jogr]

I've the next question handling post data.

 

I have created a file called 999.mailman.php in action/admin/settings

<?php
return array(
'groups' => array(
	'mailman' => array(
		'title' => $lng['admin']['mailman'],
		'fields' => array(
			'mailman_system_newlist_path' => array(
				'label' => $lng['module']['mailman']['admin']['newlist_path'],
				'settinggroup' => 'mailman',
				'varname' => 'system_newlist_path',
				'type' => 'string',
				'default' =>'',
				'save_method' => 'storeSettingField',
				),
			'mailman_system_rmlist_path' => array(
				'label' => $lng['module']['mailman']['admin']['rmlist_path'],
				'settinggroup' => 'mailman',
				'varname' => 'system_rmlist_path',
				'type' => 'string',
				'default' => '',
				'save_method' => 'storeSettingField',
				),
			),
		),
	),
);
?>

 

After that, all fields in the template are filled with the correct values taken out of the database (and the database gets filled with suggested values provided by the old script).

My next step was to change the suggested values and store them in the database.

The template code is

<tr>
<td class="main_field_name">{$lng['module']['mailman']['admin']['newlist_path']}</td>
<td class="main_field_display" nowrap="nowrap"><input type="text" name="system_newlist_path" value="{$settings['mailman']['system_newlist_path']}" /></td>
</tr>
<tr>
<td class="main_field_name">{$lng['module']['mailman']['admin']['rmlist_path']}</td>
<td class="main_field_display" nowrap="nowrap"><input type="text" name="system_rmlist_path" value="{$settings['mailman']['system_rmlist_path']}" /></td>
</tr>
<tr>
<td class="main_field_confirm" colspan="2"><input type="hidden" name="send" value="send"><input class="bottom" type="submit" value="{$lng['panel']['save']}" /></td>
</tr>

 

The code which should do the changes is if I have identified it right

$needed_commands =array("newlist","rmlist");
$suggest_commands=array("/usr/sbin/newlist");
$mysql_commands=array("system_newlist_path","system_rmlist_path");

if ( $page == 'lists' ){
 if (isset($_POST['send']) 
&& !empty($_POST['system_newlist_path']) 
&& !empty($_POST['system_rmlist_path'])) {

foreach($mysql_commands as $value){
 	$query = 'SELECT `varname`, `value` FROM `'.TABLE_PANEL_SETTINGS.'` WHERE `settinggroup` = "mailman" AND `varname`="'.$value.'"';
 	$result  = $db->query_first($query);
 	if($result && (!empty($result['value']) || isset($result['value']))){
   	$fillin_value = $_POST[$value];
	$query2 = 'UPDATE `'. TABLE_PANEL_SETTINGS .'` SET `value` = "' . $fillin_value . '" WHERE `settinggroup`="mailman" AND `varname`="'.$value.'"';
	$result2  = $db->query($query2);
 	}else{
	$query2 = 'INSERT INTO '. TABLE_PANEL_SETTINGS .' (`settinggroup`, `varname`, `value` ) VALUES ("mailman","'.$value.'","'.$_POST[$value].'")';
	$result2  = $db->query($query2);
	}
  }
}
}

 

But no changes are made when the script is called.

I don't know, if my 999.mailman.php is incorrect, or I have an error in the script routine.

[d00p]

You don't have to take care of the saving of settings, now that you have the 999.mailman.php you can edit these settings as admin on the "Settings"-page - you also don't need a template for this

[jogr]

Am I right, when I say, that if I give a default - value in 999.mailman.php this default value cannot be changed?

[d00p]

the default value given in 999.mailman.php is kind of a "backup" value if the setting cannot be found in or read from the database. You should also provide a simple SQL file with the needed CREATE TABLE's and UPDATE TABLE's according to your needs (as this should not go into Froxlor's update process, which will be overwritten on every update)

[jogr]

Ok, I was not right, but i've found my error.

[jogr]

Ok, mailman runs and it seems, that most work of migrating the old module is finished.

The old module has had a cronjob which was started by

*/5 * * * * /usr/bin/php /var/www/localhost/htdocs/syscp/scripts/modules/mailman/mailmanscript.php

 

I've tryed to run this, but I got errors from connecting to the database. As there are changes vom syscp crons to froxlor crons, I suppose this changes causes the old cron not to be able to connect to mysql.

Is there a way to add the new cron to froxlor?

 

As it might help, I will post the error message below:

~ # /usr/lib/php5/bin/php /var/www/localhost/htdocs/meinaccount/scripts/modules/mailman/mailmanscript.php
Establishing connection failed, exiting<br />
mysql error number: 1045<br />
mysql error desc: Access denied for user 'root'@'localhost' (using password: NO)<br />
Time/date: 06/10/2010 04:10 PM<br />
Script: <br />
Referer: <br />

[d00p]

Don't put anything else in the cron.d-config. Implementing new/external cronscripts requires you to insert some data into the Froxlor-database:

 

1. Move the Cronfile to the correct folder

cp /var/www/localhost/htdocs/syscp/scripts/modules/mailman/mailmanscript.php /var/www/localhost/htdocs/syscp/scripts/jobs/

 

2. Add required language-string so cron-description will be displayed in the Cronjob-settings page

(in the file where $lng['module']['mailman']['admin']['rmlist_path'] is found)

 

$lng['crondesc']['cron_mailman'] = 'Mailman cronjob';

 

3. Insert cron-information into cron-table

 

INSERT INTO 
   `cronjobs_run` 
SET
   `module` = 'mailman/mailman',
   `cronfile` = 'mailmanscript.php', 
   `interval` = '1 HOUR', 
   `isactive` = '1', 
   `desc_lng_key` = 'cron_mailman'
;

[jogr]

I've copied the mailmanscript.php file

 # cp /var/www/localhost/htdocs/meinaccount/scripts/modules/mailman/mailmanscript.php /var/www/localhost/htdocs/meinaccount/scripts/jobs/

After the cronjob has run, I've got the following error

Subject: Cron <root@myserver> /usr/lib/php5/bin/php -q /var/www/localhost/htdocs/meinaccount/scripts/froxlor_master_cronjob.php
X-Cron-Env: <PATH=/sbin:/bin:/usr/sbin:/usr/bin>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>
Message-Id: <20101007083501.BF22D2A8021@myserver.tld>
Date: Thu,  7 Oct 2010 10:35:01 +0200 (CEST)




Fatal error: Cannot redeclare class db in /var/www/localhost/htdocs/syscp/lib/class_mysqldb.php on line 24

 

I've written a howto the convert the module to froxlor. I will post it, if the module works correct.

[jogr]

I suppose, one of these lines will cause this error

 

include dirname(__FILE__) . '/../../../lib/cron_init.php/';
#include ('/var/www/syscp/lib/cron_init.php');
#include ($pathtophpfiles . '/lib/cron_init.php');

[jogr]

Ok, solved.

[d00p]

ehm, you are using the latest Froxlor version aren't you?

 

The file /lib/class_mysqldb.php doesn't even exist (for ages...)

[jogr]

Never work with more than one version of a file

 

On my server, the file looked like

<?php

$DEBUG=0;

$aliases_file = "/var/lib/mailman/data/aliases";

#include dirname(__FILE__) . '/../../../lib/cron_init.php/';
include "/var/www/localhost/htdocs/syscp/lib/cron_init.php";
#include ('/var/www/syscp/lib/cron_init.php');
#include ($pathtophpfiles . '/lib/cron_init.php');

 

The file, I've downloaded (and with which I've developed the howto) looked like

<?php

$DEBUG=0;

$aliases_file = "/var/lib/mailman/data/aliases";

include dirname(__FILE__) . '/../../../lib/cron_init.php/';
#include ('/var/www/syscp/lib/cron_init.php');
#include ($pathtophpfiles . '/lib/cron_init.php');

 

So I commented out the hard coded path und adepted the first, and it worked.

[jogr]

Aktually I am using 0.9.12

 

#include dirname(__FILE__) . '/../../../lib/cron_init.php/';

doesn't seem to work.

I have changed it to a hardcoded version

include "/var/www/localhost/htdocs/meinaccount/lib/cron_init.php";

And now I got

Fatal error: Cannot redeclare includefunctions() (previously declared in /var/www/localhost/htdocs/meinaccount/lib/functions.php:24) in /var/www/localhost/htdocs/meinaccount/lib/functions.php on line 43

[d00p]

It's still wrong anyway.

 

You do not have to include cron_init by yourself, that's what the master_cron does for you. Why don't you just look at one of the existing cronjobs to see how they work?

[jogr]

Ok, you set wrote a debughandler to your cron files.

 

I've done it to, and now I got

 

Warning: fwrite(): 6 is not a valid stream resource in /var/www/localhost/htdocs/meinaccount/scripts/froxlor_master_cronjob.php on line 27

Warning: mysql_query(): 167 is not a valid MySQL-Link resource in /var/www/localhost/htdocs/meinaccount/lib/classes/database/class.db.php on line 190

Warning: mysql_error(): 167 is not a valid MySQL-Link resource in /var/www/localhost/htdocs/meinaccount/lib/classes/database/class.db.php on line 300

Warning: mysql_errno(): 167 is not a valid MySQL-Link resource in /var/www/localhost/htdocs/meinaccount/lib/classes/database/class.db.php on line 301
Invalid SQL: SELECT MAX(`guid`) as `fguid` FROM `panel_customers`<br />
mysql error number: <br />
mysql error desc: <br />
Time/date: 07/10/2010 01:10 PM<br />
Script: <br />
Referer: <br />

 

I've changed the start of the mailmanscript.php file to

<?php

fwrite($debugHandler, 'Analysing mailman structure for possible updates' . "\n");

$DEBUG=0;

$aliases_file = "/var/lib/mailman/data/aliases";


   	if(@php_sapi_name() != 'cli' && @php_sapi_name() != 'cgi')
   	{
           	die('This script will only work in the shell.');
   	}

   	define('TABLE_MODULE_MAILMAN','modules_mailman');

/*
*  	Read in the existing lists from the system
*/

   	$cmd = $settings['mailman']['system_list_lists_path'].' -b';
   	$existing_lists = shell_exec("$cmd");
   	strtolower($existing_lists);
   	$existing_lists_array = split("\n", $existing_lists);
/*
*  	Create all new lists
*/

/*
*  	Create the List in mailman
*/
   	$result 		= $db->query("SELECT * FROM `".TABLE_MODULE_MAILMAN."` WHERE type = 'new'");
   	// foreach dataset
   	while ($row = $db->fetch_array($result)){
           	$new_created=1;
           	$id=$row['id'];
           	$list_name = substr($row['list_name'],0,strpos($row['list_name'],"@"));
           	$list_mailhost = substr($row['list_name'],strpos($row['list_name'],"@")+1);
           	$list_webhost = $row['list_webhost'];
           	$list_owner = $row['list_owner'];
           	$list_password = $row['list_password'];
           	$cmd=$settings['mailman']['system_newlist_path']." -q --urlhost=".$list_webhost." --emailhost=".$list_mailhost." ".$list_name." ".$list_owner." ".$list_password;
           	$output=shell_exec("$cmd");
           	$cmd = $settings['mailman']['system_withlist_path'].' -l -r fix_url '.$list_name.' -u '.$list_mailhost.' >/dev/null 2>&1';
           	$output = shell_exec($cmd);
   	}
   	if($new_created=="1"){
           	if(!is_file($aliases_file)){
                   	touch($aliases_file);
           	}
           	$cmd=$settings['mailman']['system_genaliases_path']." -q";
           	$output=shell_exec($cmd);
           	$aliases_file_handler = fopen($aliases_file, 'w');
           	fwrite($aliases_file_handler, $output);
           	fclose($aliases_file_handler);
           	$cmd=$settings['mailman']['system_newaliases_path'];
           	$output=shell_exec($cmd);
   	}
   	unset($new_created);

[d00p]

What are you doing? SELECT MAX(`guid`) as `fguid` FROM `panel_customers` is a SQL-query which works perfect if you run it in phpMyAdmin or the mysql-shell. Something wrong on your system. Also, i can't reproduce why the filehandle of the lock-file should be an invalid stream resource.

 

Can you please download a fresh froxlor-archive and paste a DIFF between your Froxlor-folder and the "original" so we can see all your changes

[jogr]

I've send you the link via pm.

[d00p]

Phew, you sure got a hell lot of very old files there, i recommend you follow the upgrade guide (link) and start with only recent files (the upgrade-guide will lead you there).

 

Then import your mailman-patches and then test again.

[jogr]

Ok, froxlor version is now 0.9.13.1. As I have used the "old" databese I wouldn't have done step 1 of the howto. I have done all the other steps, but changed the beginning of mailmanscript.php to beginning I have changed (and posted) 4 posts earlier.

The new .patch file is uploaded to the link, I've send you via pm.

 

The output of the mastercron is

Warning: fwrite(): 6 is not a valid stream resource in /var/www/localhost/htdocs/meinaccount/scripts/froxlor_master_cronjob.php on line 27

Warning: mysql_query(): 200 is not a valid MySQL-Link resource in /var/www/localhost/htdocs/meinaccount/lib/classes/database/class.db.php on line 190

Warning: mysql_error(): 200 is not a valid MySQL-Link resource in /var/www/localhost/htdocs/meinaccount/lib/classes/database/class.db.php on line 300

Warning: mysql_errno(): 200 is not a valid MySQL-Link resource in /var/www/localhost/htdocs/meinaccount/lib/classes/database/class.db.php on line 301
Invalid SQL: SELECT MAX(`guid`) as `fguid` FROM `panel_customers`<br />
mysql error number: <br />
mysql error desc: <br />
Time/date: 07/10/2010 02:30 PM<br />
Script: <br />
Referer: <br />

 

If I set the new mailman cronjob inactive, I will not recieve this errormessage.

[d00p]

Please nopaste the complete mailmanscript.php file - I'll try to reproduce the error

[jogr]

Here the link

http://www.nopaste.froxlor-handbuch.de/?44

[d00p]

remove include ($pathtophpfiles . '/lib/cron_shutdown.php'); (last line), i told you to check against existing cron-files, the shutdown is called by the master-cronjob

[jogr]

After deleting the last line, the cronjob works well

[atari]

Hi

 

I'll set up the new server in the next couple of days (at least i hope so). I will update the mailman module and publish it in the forum as soon as possible...

 

cheers

atari