Error 403 on apache

[Simon Yeldon]

I am having a little bit of problem getting Froxlor set up properly, with all domains that I create giving a 403 error when you access the site via http.

I am set up using the following config

• ubuntu 18.04 lts
• apache 2.4.29
• php 7.2
• php-fpm
• mod_proxy_fcgi
• libnss-extrausers

I have followed all the steps, and I think I have the correct boxes ticked, but I am obviously missing something somewhere along the lines. Any pointer in the right direction would be great.

When you go to the domain http://test.bearandbox.uk you get a 403 error with the following message

Forbidden
You don't have permission to access / on this server.
Server unable to read htaccess file, denying access to be safe

Apache/2.4.29 (Ubuntu) Server at test.bearandbox.uk Port 80

The logs give the following error

Quote

(13)Permission denied: [client XX.XXX.XX.XXX:59711] AH00529: /var/customers/webs/bearbox/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable and that '/var/customers/webs/bearbox/' is executable, referer: http://test.bearandbox.uk/

When I add a domain I get the following config in sites-available

<VirtualHost 167.99.95.176:80 [2a03:b0c0:1:e0::44a:1001]:80>
  ServerName test.bearandbox.uk
  ServerAdmin simon@bearandbox.uk
  DocumentRoot "/var/customers/webs/bearbox/test.bearandbox.uk/"
  <FilesMatch \.(php)$>
  SetHandler proxy:unix:/var/lib/apache2/fastcgi/1-bearbox-test.bearandbox.uk-php-fpm.socket|fcgi://localhost
  </FilesMatch>
  <Directory "/var/customers/webs/bearbox/test.bearandbox.uk/">
    Require all granted
    AllowOverride All
  </Directory>
  Alias /webalizer "/var/customers/webs/bearbox/webalizer"
  LogLevel warn
  ErrorLog "/var/customers/logs/bearbox-error.log"
  CustomLog "/var/customers/logs/bearbox-access.log" combined
</VirtualHost>

The file at /var/lib/extrausers looks like this

bearbox:x:10000:10000:Simon Yeldon:/var/customers/webs/bearbox/:/bin/false

the file in the php-fpm pool looks like this

;PHP-FPM configuration for "test.bearandbox.uk" created on 2019.05.21 10:30:01
[test.bearandbox.uk]
listen = /var/lib/apache2/fastcgi/1-bearbox-test.bearandbox.uk-php-fpm.socket
listen.owner = bearbox
listen.group = bearbox
listen.mode = 0660
user = bearbox
group = bearbox
pm = static
pm.max_children = 1
pm.max_requests = 0
;chroot = /var/customers/webs/bearbox/test.bearandbox.uk/
security.limit_extensions = .php
env[PATH] = /usr/local/bin:/usr/bin:/bin
env[TMP] = /var/customers/tmp/bearbox/
env[TMPDIR] = /var/customers/tmp/bearbox/
env[TEMP] = /var/customers/tmp/bearbox/
php_admin_value[session.save_path] = /var/customers/tmp/bearbox/
php_admin_value[upload_tmp_dir] = /var/customers/tmp/bearbox/


php_admin_flag[allow_call_time_pass_reference] = Off
php_admin_flag[allow_url_fopen] = Off
php_flag[asp_tags] = Off
php_admin_value[disable_classes] =
php_admin_value[disable_functions] = curl_exec,curl_multi_exec,exec,parse_ini_file,passthru,popen,proc_close,proc_get_status,proc_nice,proc_open,proc_terminate,shell_exec,show_source,system
php_flag[display_errors] = Off
php_flag[display_startup_errors] = Off
php_admin_flag[enable_dl] = Off
php_value[error_reporting] = E_ALL & ~E_NOTICE
php_admin_flag[expose_php] = Off
php_admin_flag[file_uploads] = On
php_admin_flag[cgi.force_redirect] = 1
php_admin_value[gpc_order] = "GPC"
php_flag[html_errors] = Off
php_admin_flag[ignore_repeated_errors] = Off
php_admin_flag[ignore_repeated_source] = Off
php_value[include_path] = ".:/usr/share/php/:/usr/share/php5/"
php_flag[log_errors] = On
php_admin_flag[log_errors] = On
php_value[log_errors_max_len] = 1024
php_flag[magic_quotes_gpc] = Off
php_flag[magic_quotes_runtime] = Off
php_flag[magic_quotes_sybase] = Off
php_value[max_execution_time] = 30
php_admin_value[max_input_time] = 60
php_admin_value[memory_limit] = 128M
php_admin_value[open_basedir] = "/var/customers/webs/bearbox/test.bearandbox.uk:/var/customers/tmp/bearbox:/usr/share/php:/usr/share/php5:/tmp"
php_admin_value[output_buffering] = 4096
php_admin_value[post_max_size] = 16M
php_admin_value[precision] = 14
php_admin_flag[register_argc_argv] = Off
php_admin_flag[report_memleaks] = On
php_admin_value[sendmail_path] = "/usr/sbin/sendmail -t -i -f simon@bearandbox.uk"
php_value[session.auto_start] = 0
php_value[session.cookie_domain] =
php_value[session.cookie_lifetime] = 0
php_value[session.cookie_path] = /
php_admin_value[session.gc_divisor] = 1000
php_admin_value[session.gc_probability] = 1
php_value[session.name] = PHPSESSID
php_value[session.serialize_handler] = php
php_flag[session.use_cookies] = 1
php_flag[short_open_tag] = On
php_flag[suhosin.simulation] = Off
php_flag[track_errors] = Off
php_value[upload_max_filesize] = 32M
php_admin_value[variables_order] = "GPCS"
php_admin_value[opcache.restrict_api] = "/var/customers/webs/bearbox/test.bearandbox.uk/"

If I un-comment the chroot line, it works...

How do I fix this?

[AInteriorB]

Same here. A reboot of the server solves the problem as well (restarting apache2, php-fpm and nscd is not enough). After php-fpm chroot happened once, it also works without it. Strange.

[d00p]

On 5/21/2019 at 2:42 PM, Simon Yeldon said:

(13)Permission denied: [client XX.XXX.XX.XXX:59711] AH00529: /var/customers/webs/bearbox/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable and that '/var/customers/webs/bearbox/' is executable, referer: http://test.bearandbox.uk/

error says it all, check permissions of /var/customers/webs/[user]

[AInteriorB]

vor 1 Minute schrieb d00p:

error says it all, check permissions of /var/customers/webs/[user]

Permissions are all good:

ls -la /var/customers/webs/testabcd/
drwxr-x---  6 testabcd testabcd 4096 Nov  4 13:47 .

It only happens when you setup the first domain for a brand new user. php-fpm runs as correct user, new user is listed in /var/lib/extrausers/*, folders are setup correctly, all seems good. But without chroot in php-fpm config I need a restart of the complete server, manual restarts of apache2, nscd and php-fpm are not sufficient.

[d00p]

that makes absolutely no sense - a reboot does not change setting/configs

[AInteriorB]

Indeed. I guess that nscd's invalidation doesn't work as expected thus apache2 still have old user information after reload. A complete server restart solves this of course. I figured out that this procedure also works:

nscd -i passwd
nscd -i group
/etc/init.d/apache2 restart

How can we fire nscd invalidation before apache2 and php-fpm restart in froxlor_master_cronjob.php?

[d00p]

nscd is only necessary for fcgid and fpm if not integrated via mod_proxy (required since debian 9). NSCD is being invalidated after creating a new homedir (and before webserver vhosts) and after every cronjob (when there were any tasks to complete)

[AInteriorB]

Well, setup is almost like described in Simon's initial post (fpm-php, libnss-extrausers, ...). 

After creating a new client with standard subdomain and performing

php /var/www/froxlor/scripts/froxlor_master_cronjob.php --force --debug

will lead to 403 Forbidden even after a manual apache2 restart. 

But the following solves the problem and all works as expected: 

nscd -i passwd
nscd -i group
/etc/init.d/apache2 restart

That's why I think there's a misbehaviour of my nscd invalidation. Where do you fire the nscd invalidation for froxlor_master_cronjob.php?

Thank you very much for your patience, by the way 🙂

[d00p]

25 minutes ago, AInteriorB said:

Where do you fire the nscd invalidation for froxlor_master_cronjob.php?

 

35 minutes ago, d00p said:

NSCD is being invalidated after creating a new homedir (and before webserver vhosts) and after every cronjob (when there were any tasks to complete)

 

[AInteriorB]

Yes, I have read that. But due to a successful manual invalidation I think the cron invalidation doesn't work as expected. I looked into your code

# lib/Froxlor/Cron/MasterCron.php line 137
# lib/Froxlor/Cron/System/TasksCron.php 243
                        // clear NSCD cache if using fcgid or fpm, #1570 - not needed for nss-extrausers
                        if ((\Froxlor\Settings::Get('system.mod_fcgid') == 1 || (int) \Froxlor\Settings::Get('phpfpm.enabled') == 1) && \Froxlor\Settings::Get('system.nssextrausers') == 0) {
                                $false_val = false;
                                \Froxlor\FileDir::safe_exec('nscd -i passwd 1> /dev/null', $false_val, array(
                                        '>'
                                ));
                                \Froxlor\FileDir::safe_exec('nscd -i group 1> /dev/null', $false_val, array(
                                        '>'
                                ));

Could it be, that in our setup (php-fpm and nssextrausers) we do need this invalidation there while you exclude it with your condition?

[d00p]

15 hours ago, AInteriorB said:

Could it be, that in our setup (php-fpm and nssextrausers) we do need this invalidation there while you exclude it with your condition?

nssextrausers puts passwd, groups, shadow files in /var/lib/extrausers/ and they are being included via nsswitch.conf - no, there is no need for nscd in this constellation