ripieces

Important update thanks to security researcher Ronak Nahar ( https://www.linkedin.com/in/naharronak/ ) Step 2 on h3 (secondary DNS) needs to have allow-transfer { none; } in 2.1 /etc/bind/example_bind.conf on secondary NS (h3) zone "catalog.h2.example" { type slave; file "catalog.h2.example.db"; allow-transfer { none; }; # this is new masters { <PRIMARY IP HERE>; }; }; in 2.2 /etc/bind/named.conf.options on secondary NS (h3) # ... allow-transfer { none; }; # this is new catalog-zones { zone "catalog.h2.example" default-masters { <YOUR PRIMARY IP HERE>; }; }; }; in the config, I fixed the config above accordingly.

Further reading: https://blog.thelifeofkenneth.com/2019/09/using-catalog-zones-in-bind-to.html https://bind9.readthedocs.io/en/latest/chapter6.html#namedconf-statement-catalog-zones Since you probably will have multiple servers/ catalog zones, read about how collisions can be handled: https://bind9.readthedocs.io/en/latest/chapter6.html#change-of-ownership-coo (TODO: add support for change of owner ship to script, can be as simple as adding the entries to the end of the output for the catalog file on h2 by including another file) I want to share this idea as a thank you to the Froxlor community, maybe it helps someone in the future. Notes: 1. On a Froxlor server ("primary") (called h2) that already has DNS setup: 1.1. Create script that makes the catalog zone. touch /etc/example-bind/serial chmod 600 /etc/example-bind/serial chown root:root /etc/example-bind/serial nano -w /etc/example-bind/serial 0 touch /etc/example-bind/on_reload_bind9.sh chmod 700 /etc/example-bind/on_reload_bind9.sh chown root:root /etc/example-bind/on_reload_bind9.sh nano -w /etc/example-bind/on_reload_bind9.sh #!/bin/bash catalog_name="h2" #### catalog_file="/etc/bind/catalog.${catalog_name}.example.db" touch $catalog_file chown bind:0 $catalog_file chmod 0644 $catalog_file serial=$(($(cat /etc/example-bind/serial) + 1)) echo $serial > /etc/example-bind/serial echo "catalog.${catalog_name}.example. IN SOA . . $serial 172800 960 3600000 300 catalog.${catalog_name}.example. IN NS invalid. version.catalog.${catalog_name}.example. IN TXT \"2\" " > $catalog_file named-checkconf -l /etc/bind/froxlor_bind.conf | awk "{ printf \"%s.zones.catalog.%s.example. PTR %s.\n\",makeZoneEntry(\$1),\"$catalog_name\",\$1 } function makeZoneEntry( id, cmd, output ) { cmd = \"echo \\047\" id \"\\047 | sha1sum\" if ( (cmd | getline output) > 0 ) { sub(/ .*/,\"\",output) } else { print \"failed to hash \" id | \"cat>&2\" output = id } close( cmd ) return output }" >> $catalog_file # systemctl reload bind9 # TODO: hash catalog zone and update serial only when it changes 1.2. Run the script once /etc/example-bind/on_reload_serial.sh 1.3. Create a TSIG key: tsig-keygen -a hmac-sha256 catalog.h2.example This shoud look similar to: key "catalog.h2.example" { algorithm hmac-sha256; secret "<YOUR SECRET HERE>"; }; 1.4. Create zone file for catalog zone: touch /etc/bind/example_bind.conf chmod 760 /etc/bind/example_bind.conf chown root:bind /etc/bind/example_bind.conf nano -w /etc/bind/example_bind.conf key "catalog.h2.example." { algorithm hmac-sha256; secret "<YOUR SECRET HERE>"; }; zone "catalog.h2.example" { type master; file "/etc/bind/catalog.h2.example.db"; allow-transfer { key "catalog.h2.example."; }; also-notify { <A SECONADRY IP HERE>; }; # put IPS of your DNS secondaries here }; 1.5. edit /etc/bind/named.conf.local Add # ... include "/etc/bind/example_bind.conf"; bellow the froxlor one 1.6. Test config on primary: named-checkconf -p ONLY PROCEED if it doesn't complain about errors, otherwise fix or roll-back changes. 2. On a secondary DNS (could be made with Froxlor or at least Froxlor configs with DNS setup already) (called h3): 2.1 touch /etc/bind/example_bind.conf chmod 760 /etc/bind/example_bind.conf chown root:bind /etc/bind/example_bind.conf nano -w /etc/bind/example_bind.conf key "catalog.h2.example." { algorithm "hmac-sha256"; secret "<YOUR SECRET HERE>"; }; server <PRIMARY IP HERE> { keys { "catalog.h2.example."; }; }; zone "catalog.h2.example" { type slave; file "catalog.h2.example.db"; allow-transfer { none; }; # important masters { <PRIMARY IP HERE>; }; }; 2.2 Modify /etc/bind/named.conf.options # ... allow-transfer { none; }; # important catalog-zones { zone "catalog.h2.example" default-masters { <YOUR PRIMARY IP HERE>; }; }; }; 2.3. Append to /etc/bind/named.conf.local after froxlor one: # ... include "/etc/bind/example_bind.conf"; 2.4. Test config on secondary: named-checkconf -p ONLY PROCEED if it doesn't complain about errors, otherwise fix or rollback changes. 2.5 Restart bind9 on secondary: systemctl restart bind9 3. Update Settings » Nameserver settings on primary: DNS Server reload command: /etc/example-bind/on_reload_bind9.sh IMPORTANT: Either the secondaries must be listed in Nameservers, or their IP must be in AXFR servers (Froxlor has no option for TSIG keys)

Thank you.

It redirects to https://docs.froxlor.org/apiguide/index.htmldoc/ I think you want https://docs.froxlor.org/apiguide/ maybe instead (?).

Yeah I was only worrying about admin perspective, I didn't plan adding automatic retrieval for customers. So it wouldn't be really useful for others I guess, meaning that module would be a waste of time for most people (?). Edit: Thanks for sharing those two, I didn't know them.

Is someone already developing a server-side backup module or is there one already? We are currently using some python scripts to backup web content and databases (but not mails, you would need to add that) with duplicity ( https://duplicity.gitlab.io/duplicity-web/ ) locally and then do an rclone ( https://rclone.org/ ) of that onto a remote location, but I am not quite satisfied with them for various reasons. For those interested I will share the current scripts we use since a while now as attachment, since it was lots of work and research to get them to that stage for me at least. (I think I even found some pointers for them earlier in the Froxlor forum, not sure though, don't remember). I know the files are not good and have many flaws and not good practices, but maybe it will help someone for something good. Please read the rclone and duplicity manual before asking questions (not sure if I have much time to answer them) and don't forget to install the python dependencies (I know on one server it's Python 2 and the other 3, but that's the way it is atm)! I want to switch us to BorgBackup ( https://www.borgbackup.org/ ) [it supports append-only destinations] and thought about developing a small module for Froxlor. But I would not want to do that if someone already is working on something like that and close to the finish line (but I currently can't guarantee I will actually do i). I am not sure what the best way is to start coding a Froxlor module (maybe there's s.th. in the Wiki or Forum I didn't find yet) and to be honest I haven't worked with the Froxlor API at all yet. If someone has pointers regarding these would be appreciated. Maybe someone else needs something similar and we can work together a bit somehow? I wanted the GUI (I mean the froxlor admin backend) to support: Multiple Backup locations (local, remote), including append only, and with the borg retention control options (though those are quite complicated, maybe I will just allow some commands to be added) Backup groups, that are assigned to one or multiple backup locations and have different schedules for content, mail, DB Customers can be assigned to a backup group and there can be a default backup group (not sure how to go about global backup atm) Not sure if I forgot s.th. we need for ourselves but I don't plan to go much beyond that to be honest. I do not plan to support reverse backups though ("pull mode", where a server connects to the server to backup) and also not backups on squashfs etc, I don't need it and don't consider it reliable. backup_ssh_remote.py backup.py exclude.txt global_include.txt rclone.py vgs_backup_cron_d_daily

Looking (again / more properly) at the MasterCron.php code Froxlor already uses a locking mechanism based on the PID - however I think this is doomed to fail in my opinion, since cron jobs can run in parallel with different PIDs. (this is wrong info sorry) I tried to run two scripts in parallel, one that does the nss extrausers files and one that does the chown many times each, all went through without any error, so that must be s.th. else.

froxlorlocal is the user we have for the PHP-FPM enabled froxlor-VHOST, hm.

d00p pointed this out on GitHub, this is actually a very good question / fact :S I still think it's related to libnss-extrausers, but that's really weird.

I opened an issue to track the problem on the issue tracker (I hope that is the right thing to do, I am not sure) https://github.com/Froxlor/Froxlor/issues/928 Also added notes about a solution there, but not sure if good. We could continue the discussion here to avoid spamming users that folow it there.

I switched both servers to libnss-extrausers back then for the same reason, because libnss-mysql would make things eally slow and sometimes even stall a bit. This could be indeed the reason, since up to 4 tasks can run at the same time with the default /etc/cron.d/froxlor: # automatically generated cron-configuration by froxlor # do not manually edit this file as it will be re-generated periodically. PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # */5 * * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --tasks 1> /dev/null 0 0 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --traffic 1> /dev/null 5 0 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --usage_report 1> /dev/null 0 */6 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --mailboxsize 1> /dev/null */5 * * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --letsencrypt 1> /dev/null 10 0 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --backup 1> /dev/null And each of these tasks can cause a refresh of the extrausers if more than 1 job was run: https://github.com/Froxlor/Froxlor/blob/8f850ee7f3c9339db0c09793496474fe6ab1f41c/lib/Froxlor/Cron/MasterCron.php#L114 https://github.com/Froxlor/Froxlor/blob/8f850ee7f3c9339db0c09793496474fe6ab1f41c/lib/Froxlor/Cron/MasterCron.php#L131

Sadly it didn't help on the h2 one, it just happened again Edit: Just so you can see I really entered the commands: root@h2 ~ # grep -r froxlorlocal /etc /etc/subgid-:froxlorlocal:1279648:65536 /etc/subgid:froxlorlocal:1279648:65536 /etc/gshadow-:froxlorlocal:!::www-data /etc/passwd-:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false /etc/shadow:froxlorlocal:*:17141:0:99999:7::: /etc/subuid-:froxlorlocal:1279648:65536 /etc/gshadow:www-data:*::froxlorlocal /etc/gshadow:froxlorlocal:!::www-data /etc/shadow-:froxlorlocal:*:17141:0:99999:7::: /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.owner = froxlorlocal /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.group = froxlorlocal /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:user = froxlorlocal /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:group = froxlorlocal /etc/group-:froxlorlocal:x:9999:www-data /etc/subuid:froxlorlocal:1279648:65536 /etc/passwd:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false /etc/group:www-data:x:33:froxlorlocal /etc/group:froxlorlocal:x:9999:www-data

Thank you very much for your reply. I will report back if it solved it in a few days (I have doubts and need to test it).

We have a server with many "customers" (90+, because I tend to separate websites by customers when they are not related) and we get these errors very randomly by email and with about an average of 1 - 2 times per week I think (not every day at least): Subject: Cron <root@h2> /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --tasks 1> /dev/null Time:3/7/21, 6:40 PM Content: chown: invalid user: ‘froxlorlocal:froxlorlocal’ Edit: I also had it one time already where it failed with "invalid group" instead. It's driving me a bit crazy, since I didn't find the issue yet, but it happens since quite a while, probably since after we updated to Debian 10.x (which is quite a while ago), but this also correlates with the number of "customers". We are using libnss-extrausers on the server so the bug might be related to that. We use PHP-FPM. If anyone has ideas how to go about debugging this problem, help would be appreciated 😃 I already tried / looked for several things. I am suspecting it might be a problem with libnss-extrausers or chown itself or both. On the server that has the problem: root@h2 /etc # grep -r froxlorlocal /etc /etc/subgid-:froxlorlocal:1279648:65536 /etc/subgid:froxlorlocal:1279648:65536 /etc/gshadow-:froxlorlocal:!::www-data /etc/passwd-:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false /etc/shadow:froxlorlocal:*:17141:0:99999:7::: /etc/subuid-:froxlorlocal:1279648:65536 /etc/gshadow:froxlorlocal:!::www-data /etc/shadow-:froxlorlocal:*:17141:0:99999:7::: /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.owner = froxlorlocal /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.group = froxlorlocal /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:user = froxlorlocal /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:group = froxlorlocal /etc/group-:froxlorlocal:x:9999:www-data /etc/subuid:froxlorlocal:1279648:65536 /etc/passwd:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false /etc/group:froxlorlocal:x:9999:www-data On the second server that has very similar setup, but way less customers and never has that error: root@h3:~# grep -r froxlorlocal /etc /etc/shadow-:froxlorlocal:*:17990:0:99999:7::: /etc/subuid:froxlorlocal:100000:65536 /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:listen.owner = froxlorlocal /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:listen.group = froxlorlocal /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:user = froxlorlocal /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:group = froxlorlocal /etc/group-:froxlorlocal:x:9999:www-data /etc/subgid:froxlorlocal:100000:65536 /etc/gshadow:froxlorlocal:!::www-data /etc/shadow:froxlorlocal:*:17990:0:99999:7::: /etc/gshadow-:froxlorlocal:!::www-data /etc/subuid-:froxlorlocal:100000:65536 /etc/subgid-:froxlorlocal:100000:65536 /etc/passwd-:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false /etc/passwd:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false /etc/group:froxlorlocal:x:9999:www-data

For one of my friend's company this _could_ be interesting, if you have proper backups / depending on how those are handled. This is the biggest thing that keeps us from moving to other solutions, currently we use duplicity to backup encrypted on some cloud service(s) and moving elsewhere would mean usually a lot of drawbacks on the backups / backup retention time etc. But just see it as note, maybe many other customers have other problems / priorities and this won't be one with them maybe. Anyways: Good luck and success with your new project!