Jump to content
Froxlor Forum

ripieces

Members
  • Posts

    20
  • Joined

  • Last visited

  • Days Won

    1

Posts posted by ripieces

  1. Important update thanks to security researcher Ronak Nahar ( https://www.linkedin.com/in/naharronak/ )

    Step 2 on h3 (secondary DNS) needs to have allow-transfer { none; }

    in 2.1 /etc/bind/example_bind.conf on secondary NS (h3)

    zone "catalog.h2.example" {
            type slave;
            file "catalog.h2.example.db";
            allow-transfer { none; }; # this is new
            masters { <PRIMARY IP HERE>; };
    };

    in 2.2 /etc/bind/named.conf.options on secondary NS (h3)

    # ...
            allow-transfer { none; }; # this is new
            
            catalog-zones {
                    zone "catalog.h2.example" default-masters { <YOUR PRIMARY IP HERE>; };
            };
    };

    in the config, I fixed the config above accordingly.

  2. Further reading:


    I want to share this idea as a thank you to the Froxlor community, maybe it helps someone in the future.

    Notes:

    1. On a Froxlor server ("primary") (called h2) that already has DNS setup:

    1.1. Create script that makes the catalog zone.

    touch /etc/example-bind/serial
    chmod 600 /etc/example-bind/serial
    chown root:root /etc/example-bind/serial
    nano -w /etc/example-bind/serial

    0

    touch /etc/example-bind/on_reload_bind9.sh
    chmod 700 /etc/example-bind/on_reload_bind9.sh
    chown root:root /etc/example-bind/on_reload_bind9.sh
    nano -w /etc/example-bind/on_reload_bind9.sh

     

    #!/bin/bash
    
    catalog_name="h2"
    
    ####
    
    catalog_file="/etc/bind/catalog.${catalog_name}.example.db"
    
    touch $catalog_file
    chown bind:0 $catalog_file
    chmod 0644 $catalog_file
    
    serial=$(($(cat /etc/example-bind/serial) + 1))
    echo $serial > /etc/example-bind/serial
    
    echo "catalog.${catalog_name}.example.    IN SOA . . $serial 172800 960 3600000 300
    catalog.${catalog_name}.example.    IN NS invalid.
    version.catalog.${catalog_name}.example.    IN TXT \"2\"
    " > $catalog_file
    
    named-checkconf -l /etc/bind/froxlor_bind.conf | awk "{ printf \"%s.zones.catalog.%s.example. PTR %s.\n\",makeZoneEntry(\$1),\"$catalog_name\",\$1 }
    function makeZoneEntry( id,  cmd, output ) {
            cmd = \"echo \\047\" id \"\\047 | sha1sum\"
            if ( (cmd | getline output) > 0 ) {
                    sub(/ .*/,\"\",output)
            }
            else {
                    print \"failed to hash \" id | \"cat>&2\"
                    output = id
            }
            close( cmd )
            return output
    }" >> $catalog_file
    
    #
    systemctl reload bind9

    # TODO: hash catalog zone and update serial only when it changes


    1.2. Run the script once

    /etc/example-bind/on_reload_serial.sh

    1.3. Create a TSIG key:

    tsig-keygen -a hmac-sha256 catalog.h2.example

    This shoud look similar to:

    key "catalog.h2.example" {
        algorithm hmac-sha256;
        secret "<YOUR SECRET HERE>";
    };

    1.4. Create zone file for catalog zone:

    touch /etc/bind/example_bind.conf
    chmod 760 /etc/bind/example_bind.conf
    chown root:bind /etc/bind/example_bind.conf
    nano -w /etc/bind/example_bind.conf

    key "catalog.h2.example." {
            algorithm hmac-sha256;
            secret "<YOUR SECRET HERE>";
    };
    
    zone "catalog.h2.example" {
            type master;
            file "/etc/bind/catalog.h2.example.db";
            allow-transfer   { key "catalog.h2.example."; };
            also-notify { <A SECONADRY IP HERE>; }; # put IPS of your DNS secondaries here
    };


    1.5. edit /etc/bind/named.conf.local

    Add

    # ...
    
    include "/etc/bind/example_bind.conf";

    bellow the froxlor one

    1.6. Test config on primary:

    named-checkconf -p
    ONLY PROCEED if it doesn't complain about errors, otherwise fix or roll-back changes.

     

    2. On a secondary DNS (could be made with Froxlor or at least Froxlor configs with DNS setup already) (called h3):

    2.1

    touch /etc/bind/example_bind.conf
    chmod 760 /etc/bind/example_bind.conf
    chown root:bind /etc/bind/example_bind.conf
    nano -w /etc/bind/example_bind.conf

    key "catalog.h2.example." {
            algorithm "hmac-sha256";
            secret "<YOUR SECRET HERE>";
    };
    server <PRIMARY IP HERE> {
            keys { "catalog.h2.example."; };
    };
    
    zone "catalog.h2.example" {
            type slave;
            file "catalog.h2.example.db";
            allow-transfer { none; }; # important
            masters { <PRIMARY IP HERE>; };
    };
    

    2.2 Modify /etc/bind/named.conf.options

    # ...
            allow-transfer { none; }; # important
            
            catalog-zones {
                    zone "catalog.h2.example" default-masters { <YOUR PRIMARY IP HERE>; };
            };
    };


    2.3. Append to /etc/bind/named.conf.local after froxlor one:

    # ...
    
    include "/etc/bind/example_bind.conf";

    2.4. Test config on secondary:
    named-checkconf -p
    ONLY PROCEED if it doesn't complain about errors, otherwise fix or rollback changes.

    2.5 Restart bind9 on secondary:

    systemctl restart bind9

    3. Update Settings  » Nameserver settings on primary:

    • DNS Server reload command: /etc/example-bind/on_reload_bind9.sh
    • IMPORTANT: Either the secondaries must be listed in Nameservers, or their IP must be in AXFR servers (Froxlor has no option for TSIG keys)
  3. Is someone already developing a server-side backup module or is there one already?

    We are currently using some python scripts to backup web content and databases (but not mails, you would need to add that) with duplicity ( https://duplicity.gitlab.io/duplicity-web/ ) locally and then do an rclone ( https://rclone.org/ ) of that onto a remote location, but I am not quite satisfied with them for various reasons.

    For those interested I will share the current scripts we use since a while now as attachment, since it was lots of work and research to get them to that stage for me at least. (I think I even found some pointers for them earlier in the Froxlor forum, not sure though, don't remember).
    I know the files are not good and have many flaws and not good practices, but maybe it will help someone for something good.
    Please read the rclone and duplicity manual before asking questions (not sure if I have much time to answer them) and don't forget to install the python dependencies (I know on one server it's Python 2 and the other 3, but that's the way it is atm)!

     

    I want to switch us to BorgBackup ( https://www.borgbackup.org/ ) [it supports append-only destinations] and thought about developing a small module for Froxlor.
    But I would not want to do that if someone already is working on something like that and close to the finish line (but I currently can't guarantee I will actually do i).
    I am not sure what the best way is to start coding a Froxlor module (maybe there's s.th. in the Wiki or Forum I didn't find yet) and to be honest I haven't worked with the Froxlor API at all yet. If someone has pointers regarding these would be appreciated.
    Maybe someone else needs something similar and we can work together a bit somehow?

    I wanted the GUI (I mean the froxlor admin backend) to support:

    • Multiple Backup locations (local, remote), including append only, and with the borg retention control options (though those are quite complicated, maybe I will just allow some commands to be added)
    • Backup groups, that are assigned to one or multiple backup locations and have different schedules for content, mail, DB
    • Customers can be assigned to a backup group and there can be a default backup group (not sure how to go about global backup atm)

    Not sure if I forgot s.th. we need for ourselves but I don't plan to go much beyond that to be honest.

    I do not plan to support reverse backups though ("pull mode", where a server connects to the server to backup) and also not backups on squashfs etc, I don't need it and don't consider it reliable.



     

    backup_ssh_remote.py backup.py exclude.txt global_include.txt rclone.py vgs_backup_cron_d_daily

  4. Looking (again / more properly) at the MasterCron.php code Froxlor already uses a locking mechanism based on the PID - however I think this is doomed to fail in my opinion, since cron jobs can run in parallel with different PIDs. (this is wrong info sorry)

    I tried to run two scripts in parallel, one that does the nss extrausers files and one that does the chown many times each, all went through without any error, so that must be s.th. else.

  5. 21 hours ago, rseffner said:

    Same here using libnss-extrausers. So this should only happen in moments cron is running WHILE froxlor is writing new passwd/groups to /var/lib/libextrausers.

    But why happens froxlor jobs in parallel (writing extrausers and doing someting in scope of these users).

    Maybe it shoud not be:
    - deleting files
    - generating files new from SQL using a query (consuming some time)

    instead
    - generating files new from SQL using a query as FILE.NEW
    - delete old ones and rename/move FILE.NEW

    or
    - nscd may help (I removed this with migrating libnss-mysql to libnss-extrausers becauso of seldom problems nscd used 100% CPU)

    I switched both servers to libnss-extrausers back then for the same reason, because libnss-mysql would make things eally slow and sometimes even stall a bit.


    This could be indeed the reason, since up to 4 tasks can run at the same time with the default /etc/cron.d/froxlor:
     

    # automatically generated cron-configuration by froxlor
    # do not manually edit this file as it will be re-generated periodically.
    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    #
    */5 * * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --tasks 1> /dev/null
    0 0 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --traffic 1> /dev/null
    5 0 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --usage_report 1> /dev/null
    0 */6 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --mailboxsize 1> /dev/null
    */5 * * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --letsencrypt 1> /dev/null
    10 0 * * * root /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --backup 1> /dev/null
    

    And each of these tasks can cause a refresh of the extrausers if more than 1 job was run:
    https://github.com/Froxlor/Froxlor/blob/8f850ee7f3c9339db0c09793496474fe6ab1f41c/lib/Froxlor/Cron/MasterCron.php#L114
    https://github.com/Froxlor/Froxlor/blob/8f850ee7f3c9339db0c09793496474fe6ab1f41c/lib/Froxlor/Cron/MasterCron.php#L131

  6. Sadly it didn't help on the h2 one, it just happened again :(

    Edit: Just so you can see I really entered the commands:

    root@h2 ~ # grep -r froxlorlocal /etc
    /etc/subgid-:froxlorlocal:1279648:65536
    /etc/subgid:froxlorlocal:1279648:65536
    /etc/gshadow-:froxlorlocal:!::www-data
    /etc/passwd-:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false
    /etc/shadow:froxlorlocal:*:17141:0:99999:7:::
    /etc/subuid-:froxlorlocal:1279648:65536
    /etc/gshadow:www-data:*::froxlorlocal
    /etc/gshadow:froxlorlocal:!::www-data
    /etc/shadow-:froxlorlocal:*:17141:0:99999:7:::
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.owner = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.group = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:user = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:group = froxlorlocal
    /etc/group-:froxlorlocal:x:9999:www-data
    /etc/subuid:froxlorlocal:1279648:65536
    /etc/passwd:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false
    /etc/group:www-data:x:33:froxlorlocal
    /etc/group:froxlorlocal:x:9999:www-data
    

     

  7. We have a server with many "customers" (90+, because I tend to separate websites by customers when they are not related) and we get these errors very randomly by email and with about an average of 1  - 2 times per week I think (not every day at least):

    Subject: Cron <root@h2> /usr/bin/nice -n 5 /usr/bin/php -q /var/www/froxlor/scripts/froxlor_master_cronjob.php --tasks 1> /dev/null
    Time:3/7/21, 6:40 PM

    Content:

    chown: invalid user: ‘froxlorlocal:froxlorlocal’
    

    Edit: I also had it one time already where it failed with "invalid group" instead.


    It's driving me a bit crazy, since I didn't find the issue yet, but it happens since quite a while, probably since after we updated to Debian 10.x (which is quite a while ago), but this also correlates with the number of "customers".
    We are using libnss-extrausers on the server so the bug might be related to that.
    We use PHP-FPM.

    If anyone has ideas how to go about debugging this problem, help would be appreciated 😃
    I already tried / looked for several things. I am suspecting it might be a problem with libnss-extrausers or chown itself or both.

    On the server that has the problem:

    root@h2 /etc # grep -r froxlorlocal /etc
    /etc/subgid-:froxlorlocal:1279648:65536
    /etc/subgid:froxlorlocal:1279648:65536
    /etc/gshadow-:froxlorlocal:!::www-data
    /etc/passwd-:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false
    /etc/shadow:froxlorlocal:*:17141:0:99999:7:::
    /etc/subuid-:froxlorlocal:1279648:65536
    /etc/gshadow:froxlorlocal:!::www-data
    /etc/shadow-:froxlorlocal:*:17141:0:99999:7:::
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.owner = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:listen.group = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:user = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h2.vgstudios.co.il.conf:group = froxlorlocal
    /etc/group-:froxlorlocal:x:9999:www-data
    /etc/subuid:froxlorlocal:1279648:65536
    /etc/passwd:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false
    /etc/group:froxlorlocal:x:9999:www-data

    On the second server that has very similar setup, but way less customers and never has that error:

    root@h3:~# grep -r froxlorlocal /etc
    /etc/shadow-:froxlorlocal:*:17990:0:99999:7:::
    /etc/subuid:froxlorlocal:100000:65536
    /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:listen.owner = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:listen.group = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:user = froxlorlocal
    /etc/php/7.3/fpm/pool.d/h3.vagas.co.il.conf:group = froxlorlocal
    /etc/group-:froxlorlocal:x:9999:www-data
    /etc/subgid:froxlorlocal:100000:65536
    /etc/gshadow:froxlorlocal:!::www-data
    /etc/shadow:froxlorlocal:*:17990:0:99999:7:::
    /etc/gshadow-:froxlorlocal:!::www-data
    /etc/subuid-:froxlorlocal:100000:65536
    /etc/subgid-:froxlorlocal:100000:65536
    /etc/passwd-:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false
    /etc/passwd:froxlorlocal:x:9999:9999:,,,:/home/froxlorlocal:/bin/false
    /etc/group:froxlorlocal:x:9999:www-data

     

  8. For one of my friend's company this _could_ be interesting, if you have proper backups / depending on how those are handled.
    This is the biggest thing that keeps us from moving to other solutions, currently we use duplicity to backup encrypted on some cloud service(s) and moving elsewhere would mean usually a lot of drawbacks on the backups / backup retention time etc.

    But just see it as note, maybe many other customers have other problems / priorities and this won't be one with them maybe.

    Anyways: Good luck and success with your new project!

    • Thanks 1
  9. Yes sorry, I meant 0.10.17-1 (current latest).

    [Tue 26 May 2020 12:07:10 AM CEST] 3155.co.il:Verify error:Invalid response from https://3155.co.il/.well-known/acme-challenge/iySt9DPeykV5--MPZDoEbhBcdRhvDERKKhybpcJk7oo [2606:4700:3036::681f:4e4a]: 
    [Tue 26 May 2020 12:07:10 AM CEST] Please add '--debug' or '--log' to check more details.
    [Tue 26 May 2020 12:07:10 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Tue 26 May 2020 12:07:11 AM CEST] Error renew 3155.co.il.
    [Tue 26 May 2020 12:07:20 AM CEST] hasade.vagas.co.il:Verify error:DNS problem: NXDOMAIN looking up A for hasade.vagas.co.il - check that a DNS record exists for this domain
    [Tue 26 May 2020 12:07:20 AM CEST] Please add '--debug' or '--log' to check more details.
    [Tue 26 May 2020 12:07:20 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Tue 26 May 2020 12:07:21 AM CEST] Error renew hasade.vagas.co.il.
    [Tue 26 May 2020 12:07:29 AM CEST] hibur-ins.co.il:Verify error:Invalid response from http://hibur-ins.co.il/.well-known/acme-challenge/WWxo_yP7fATDcZu1kpf7ejm3YIoXhKieZupJ-uFrmuI [88.218.117.67]: 
    [Tue 26 May 2020 12:07:29 AM CEST] Please add '--debug' or '--log' to check more details.
    [Tue 26 May 2020 12:07:29 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Tue 26 May 2020 12:07:30 AM CEST] Error renew hibur-ins.co.il.
    [Tue 26 May 2020 12:07:45 AM CEST] lerougeparis.com:Verify error:DNS problem: NXDOMAIN looking up A for lerougeparis.com - check that a DNS record exists for this domain
    [Tue 26 May 2020 12:07:45 AM CEST] Please add '--debug' or '--log' to check more details.
    [Tue 26 May 2020 12:07:45 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Tue 26 May 2020 12:07:50 AM CEST] Error renew lerougeparis.com.
    
    [Wed 27 May 2020 12:07:12 AM CEST] 3155.co.il:Verify error:Invalid response from https://3155.co.il/.well-known/acme-challenge/MCobMUL5wTPs-XCoir5oTSihu4ye3Guac5Kviz6FQ-c [2606:4700:3036::681f:4e4a]: 
    [Wed 27 May 2020 12:07:12 AM CEST] Please add '--debug' or '--log' to check more details.
    [Wed 27 May 2020 12:07:12 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Wed 27 May 2020 12:07:14 AM CEST] Error renew 3155.co.il.
    [Wed 27 May 2020 12:07:23 AM CEST] hasade.vagas.co.il:Verify error:DNS problem: NXDOMAIN looking up A for hasade.vagas.co.il - check that a DNS record exists for this domain
    [Wed 27 May 2020 12:07:23 AM CEST] Please add '--debug' or '--log' to check more details.
    [Wed 27 May 2020 12:07:23 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Wed 27 May 2020 12:07:23 AM CEST] Error renew hasade.vagas.co.il.
    [Wed 27 May 2020 12:07:36 AM CEST] hibur-ins.co.il:Verify error:Invalid response from http://hibur-ins.co.il/.well-known/acme-challenge/J14Nx0fTR-91ZK4bR02twD1wk-KwyU0ptaeTVCCv_DQ [88.218.117.67]: 
    [Wed 27 May 2020 12:07:36 AM CEST] Please add '--debug' or '--log' to check more details.
    [Wed 27 May 2020 12:07:36 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Wed 27 May 2020 12:07:37 AM CEST] Error renew hibur-ins.co.il.
    [Wed 27 May 2020 12:07:49 AM CEST] lerougeparis.com:Verify error:DNS problem: NXDOMAIN looking up A for lerougeparis.com - check that a DNS record exists for this domain
    [Wed 27 May 2020 12:07:49 AM CEST] Please add '--debug' or '--log' to check more details.
    [Wed 27 May 2020 12:07:49 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Wed 27 May 2020 12:07:52 AM CEST] Error renew lerougeparis.com.
    
    [Thu 28 May 2020 12:07:10 AM CEST] 3155.co.il:Verify error:Invalid response from https://3155.co.il/.well-known/acme-challenge/axNKAqQ64Giv_34HhADn71RDTXxqz0Wgu7HRyO8QlhY [2606:4700:3031::681f:4f4a]: 
    [Thu 28 May 2020 12:07:10 AM CEST] Please add '--debug' or '--log' to check more details.
    [Thu 28 May 2020 12:07:10 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Thu 28 May 2020 12:07:13 AM CEST] Error renew 3155.co.il.
    [Thu 28 May 2020 12:07:22 AM CEST] hasade.vagas.co.il:Verify error:DNS problem: NXDOMAIN looking up A for hasade.vagas.co.il - check that a DNS record exists for this domain
    [Thu 28 May 2020 12:07:22 AM CEST] Please add '--debug' or '--log' to check more details.
    [Thu 28 May 2020 12:07:22 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Thu 28 May 2020 12:07:23 AM CEST] Error renew hasade.vagas.co.il.
    [Thu 28 May 2020 12:07:31 AM CEST] hibur-ins.co.il:Verify error:Invalid response from http://hibur-ins.co.il/.well-known/acme-challenge/uPHrDnThy_o4SkeAvV_Az1qVz-4k0vzNf_pgtVCXeK8 [88.218.117.67]: 
    [Thu 28 May 2020 12:07:31 AM CEST] Please add '--debug' or '--log' to check more details.
    [Thu 28 May 2020 12:07:31 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Thu 28 May 2020 12:07:33 AM CEST] Error renew hibur-ins.co.il.
    [Thu 28 May 2020 12:07:44 AM CEST] lerougeparis.com:Verify error:DNS problem: NXDOMAIN looking up A for lerougeparis.com - check that a DNS record exists for this domain
    [Thu 28 May 2020 12:07:44 AM CEST] Please add '--debug' or '--log' to check more details.
    [Thu 28 May 2020 12:07:44 AM CEST] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
    [Thu 28 May 2020 12:07:47 AM CEST] Error renew lerougeparis.com.
    

    3155.co.il - We don't host ourselves since many months and removed it from the panel.
    hasade.vagas.co.il - was removed long time ago for now.
    hibur-ins.co.il - same as 3155.co.il.
    Lerougeparis.com - we removed this from SSL port (but maybe I also need to remove it from Let's Encrypt, so might be partially our fault for this one?).

×
×
  • Create New...