Error 502: Bad gateway when sending e-mail to Group members

When sending an e-mail to all members of a "Group", the software gets in a "Error 502: Bad Gateway".

My configuraion is the following:

Elgg version 4.3.4

PHP version: 7.4.33
PHP extension: pdo_mysql Enabled
PHP extension: json Enabled
PHP extension: xml Enabled
PHP extension: gd Enabled
PHP extension: mbstring Enabled
Session Garbage Collection Disabled
If garbage collection is not configured the session table will not be cleaned up. Configure session.gc_divisor and session.gc_probability in your php.ini.
Database server mariadb v10.3.36-MariaDB-0+deb10u2
Database client PDO MySQL Driver
.htaccess rewrite rules OK

Server     nginx/1.14.2

PHP version     7.4.33
PHP ini file location     /etc/php/7.4/fpm/php.ini
PHP Log     Web server's error log
Memory available     134,217,728
POST maximum size     8,388,608
Upload maximum size     2,097,152 

Log of the operations leading to the error:

 

 

2023/01/27 14:53:45 [error] 6591#6591: *2419 FastCGI sent in stderr: "PHP message: [2023-01-27T14:53:44.000954+01:00] ELGG.INFO: DB query results returned fro
m cache (hash: 2736445645cb8ba51b42683d3557ade1)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip"
:"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:44.001062+01:00] ELGG.INFO: DB select query SELECT * FROM elgg_users_sessions WHERE session = :qb1 (params: Array
(
   [qb1] => cdl26djceg1fegtjemerf9qo2q
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.286914+01:00] ELGG.INFO: DB query results cached (hash: 2736445645cb8ba51b42683d3557ade1)  {"process_id":6594,"memory_peak_u
sage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site"
,"referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.288567+01:00] ELGG.INFO: DB query results returned from cache (hash: 604114637a252a41804774952d6fc9d9)  {"process_id":6594,"
memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliti
calcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.288747+01:00] ELGG.INFO: DB select query SELECT e.* FROM elgg_entities e WHERE ((e.enabled = :qb2) AND (e.access_id IN (:qb1
))) AND (e.guid = :qb3) (params: Array
(
   [qb1] => 2
   [qb2] => yes
   [qb3] => 547
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB",
2023/01/27 14:53:45 [error] 6591#6591: *2419 FastCGI sent in stderr: "ND (e.type = :qb2)) AND ((r.relationship IN (:qb3)) AND (r.guid_two IN (:qb4))) ORDER BY
e.time_created desc, e.guid desc (params: Array
(
   [qb1] => yes
   [qb2] => user
   [qb3] => member
   [qb4] => 107
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.319573+01:00] ELGG.INFO: DB query results cached (hash: 69b9e1c85221cdde878b19db7cc179e4)  {"process_id":6594,"memory_peak_u
sage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site"
,"referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.320705+01:00] ELGG.INFO: DB insert query INSERT INTO elgg_entities (type, subtype, owner_guid, container_guid, access_id, ti
me_created, time_updated, last_action) VALUES(:qb1, :qb2, :qb3, :qb4, :qb5, :qb6, :qb7, :qb8) (params: Array
(
   [qb1] => object
   [qb2] => group_tools_group_mail
   [qb3] => 547
   [qb4] => 107
   [qb5] => 2
   [qb6] => 1674827625
   [qb7] => 1674827625
   [qb8] => 1674827625
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.320906+01:00] ELGG.INFO: Query cache invalidated  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","ur
l":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoli
ticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.325580+
2023/01/27 14:53:45 [error] 6591#6591: *2419 FastCGI sent in stderr: ")
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.335270+01:00] ELGG.INFO: Query cache invalidated  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","ur
l":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoli
ticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.340352+01:00] ELGG.INFO: DB query results returned from cache (hash: 6a90a81e7cafb0a42e53b5ba759958b6)  {"process_id":6594,"
memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliti
calcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.340544+01:00] ELGG.INFO: DB select query SELECT e.* FROM elgg_entities e WHERE e.guid = :qb1 (params: Array
(
   [qb1] => 769
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.340997+01:00] ELGG.INFO: DB query results cached (hash: 6a90a81e7cafb0a42e53b5ba759958b6)  {"process_id":6594,"memory_peak_u
sage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site"
,"referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.343590+01:00] ELGG.INFO: DB query results returned from cache (hash: 6a90a81e7cafb0a42e53b5ba759958b
2023/01/27 14:53:45 [error] 6591#6591: *2419 FastCGI sent in stderr: "ussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/g
roups/mail/107"}
PHP message: [2023-01-27T14:53:45.357516+01:00] ELGG.INFO: DB query results returned from cache (hash: 6a90a81e7cafb0a42e53b5ba759958b6)  {"process_id":6594,"
memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliti
calcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.357611+01:00] ELGG.INFO: DB select query SELECT e.* FROM elgg_entities e WHERE e.guid = :qb1 (params: Array
(
   [qb1] => 769
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.357899+01:00] ELGG.INFO: DB query results cached (hash: 6a90a81e7cafb0a42e53b5ba759958b6)  {"process_id":6594,"memory_peak_u
sage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":"discussion.cosmopoliticalcoop.site"
,"referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.358189+01:00] ELGG.INFO: DB update query UPDATE elgg_entities SET last_action = :qb1 WHERE guid = :qb2 (params: Array
(
   [qb1] => 1674827625
   [qb2] => 107
)
)  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","url":"/action/group_tools/mail","ip":"88.125.101.84","http_method":"POST","server":
"discussion.cosmopoliticalcoop.site","referrer":"https://discussion.cosmopoliticalcoop.site/groups/mail/107"}
PHP message: [2023-01-27T14:53:45.358252+01:00] ELGG.INFO: Query cache invalidated  {"process_id":6594,"memory_peak_usage":"4.03 MB","memory_usage":"2 MB","ur
l":"/action/group_tools/m
2023/01/27 14:53:46 [error] 6591#6591: *2419 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 88.125.101.84,
server: discussion.cosmopoliticalcoop.site, request: "POST /action/group_tools/mail HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host
: "discussion.cosmopoliticalcoop.site", referrer: "https://discussion.cosmopoliticalcoop.site/groups/mail/107"

  • recv() failed (104: Connection reset by peer) while reading response header from upstream

    In most cases, this error is related to Nginx configuration.

    Just Google it, there's a lot of answers on this.

  • Thank you @nikolai ! This is a very helpful hint. I will investigate and let you know what comes out of it.

     

  • I have tested the solution proposed by this source, which addresses this specific nginx server error: https://www.digitalreborn.com/fix-nginx-connection-reset-by-peer-upstream/

    but the "Bad gateway - 502" error is still there. :-(

  • Sorry, I can't help you without knowing your server configuration or access to it ¯\_(ツ)_/¯

  • Thank you for having looked into this. I had attempted to provide some elements of server configuation in my first post.

    Elgg version 4.3.4

    PHP version: 7.4.33
    PHP extension: pdo_mysql Enabled
    PHP extension: json Enabled
    PHP extension: xml Enabled
    PHP extension: gd Enabled
    PHP extension: mbstring Enabled
    Session Garbage Collection Disabled
    If garbage collection is not configured the session table will not be cleaned up. Configure session.gc_divisor and session.gc_probability in your php.ini.
    Database server mariadb v10.3.36-MariaDB-0+deb10u2
    Database client PDO MySQL Driver
    .htaccess rewrite rules OK

    Server     nginx/1.14.2

    PHP version     7.4.33
    PHP ini file location     /etc/php/7.4/fpm/php.ini
    PHP Log     Web server's error log
    Memory available     134,217,728
    POST maximum size     8,388,608
    Upload maximum size     2,097,152 

    I would be ready to grant you access to the server, but would need more information regarding you and your organisation. Where are you located? Do you act as a physical person or as a legal entity, and if so, what is it and where is it registered? I would also be eager to have a telephone conversation with you to understand better your relation to Elgg, as I understand that you are developping your own social app platform. I would also be interested to know if you would want to be remunerated for this service, and if so, how much.

    I am located in France. The website in question is intended to be the internal social network of a European cooperative currently being created, in a "private community" configuration.

  • I've meant Nginx config ;)

    Start to test Nginx:

    nginx -t

    Try raising the timeout:

    location ~ .php$ {
         ......
         fastcgi_read_timeout 300;
         ......
    }

    In /etc/php/7.4/fpm/pool.d/www.conf

    change

    ;request_terminate_timeout = 0

    on

    request_terminate_timeout = 300
    Maybe you need to change the process manager (or its values) in /etc/php/7.4/fpm/pool.d/www.conf 
     
    Don't forget to restart Nginx and PHP-FPM after changing settings:
    service nginx reload

    service php-7.4-fpm restart

    It's important to understand that it is highly desirable to get rid of slow scripts. A solution with timeout settings may stop working when the load increases.
     
    I recommend also enabling Session Garbage Collection in your post above.
  • Thank you Nikolai! I will test this and come back to you.

    I am sorry that I am slow to respond. This is a volunteer project that I can only work on 1 day / week on Fridays.

  • Your suggestion was tested, but the outcome remains the same:

    2023/02/17 13:52:00 [error] 14614#14614: *1 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 88.165.95.156, server: discussion.cosmopoliticalcoop.site, request: "POST /action/group_tools/mail HTTP/2.0", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "discussion.cosmopoliticalcoop.site", referrer: "https://discussion.cosmopoliticalcoop.site/groups/mail/107"