We had a WooCommerce Checkout Sendgrid Issue on one of our Trellis servers. Payments did work, but no feedback was sent to client or very late. No confirmation of successful sale was given. This is very inconvenient obviously so we checked out what was the issue and solution. We soon found out WooCommerce and Sendgrid were not plating nice. Here below the whole discovery process.
Upstream Timed Out
The error we had was:
2018/01/25 08:27:10 [error] 16241#16241: *42582 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 213.165.184.36, server: domain.com, request: "POST /?wc-ajax=checkout HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm-wordpress.sock", host: "domain.com", referrer: "https://domain.com/checkout/"
Port 110 is the post office protocol port and the ip address 213.165.184.36 an address of a Malta Cable company. Not much to go on early other than that there seems to be a time-out issue and that Nginx could perhaps use some more Ks for its buffer.
Nginx Buffering
So we decided to up the Nginx buffer using:
nginx_fastcgi_buffers: 16 16k nginx_fastcgi_buffer_size: 32k
inside group/vars/production/main.yml. This I added and re-provisioned our Trellis server.
Sendgrid
The other thing we wondered about if Sendgrid was having issues sending out details after a successful Stripe payment had been made. We were after all using it for outgoing emails using:
Documentation: https://roots.io/trellis/docs/mail/
mail_smtp_server: smtp.sendgrid.net:587
mail_admin: admin@publiqly.com
mail_hostname: publiqly.com
mail_user: publiqly
mail_password: "{{ vault_mail_password }}" # Define this variable in group_vars/all/vault.yml
When we checked Sendgrid we hardly saw any traffic. Something to worry about.
WP Mail Logging & sSMTP Logging
So we decided to install WP Mail Logging to facilitate the checking of all outgoing email. We also activated sSMTP mail logging. This you can do by setting
Debug=Yes
in ssmtp.conf and then check syslog for any errors.
Mail Logs
And then I thought about the standard mail logs. And when I checked at /var/log/mail.err I found:
Jan 25 08:29:28 domain sSMTP[16416]: Cannot open smtp.sendgrid.net:587 Jan 25 08:33:42 domain sSMTP[16424]: Unable to connect to "smtp.sendgrid.net" port 587. Jan 25 08:33:42 domain sSMTP[16424]: Cannot open smtp.sendgrid.net:587 Jan 25 09:07:42 domain sSMTP[16603]: Unable to connect to "smtp.sendgrid.net" port 587. Jan 25 09:07:42 domain sSMTP[16603]: Cannot open smtp.sendgrid.net:587
Well there you go. It seems the connection cannot be made properly. I contacted Sendgrid one this.
Port 587
Found out port 587 like most ports aren’t open on Trellis. This by doing a:
# netstat -ntlp | grep LISTEN tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 1500/nginx -g daemo tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 1343/memcached tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1500/nginx -g daemo tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 23307/sshd tcp6 0 0 :::443 :::* LISTEN 1500/nginx -g daemo tcp6 0 0 :::3306 :::* LISTEN 1618/mysqld tcp6 0 0 :::80 :::* LISTEN 1500/nginx -g daemo
So based on a Roots forum search I added:
- type: dport_accept dport: [587] protocol: tcp - type: dport_accept dport: [587] protocol: udp
to group_vars/all/security.yaml. Then I re-provisioned those playbooks:
ansible-playbook server.yml --tags "ferm,ssmtp, mail" -e env=production
Ports not the Issue
Then based on the Roots Discourse thread I had running I realized we were talking outgoing port. It is not incoming traffic that is the issue. And that the issue was more with Sendgrid or the way Sendgrid dealt with the incoming requests. SSH and https/http ports are listening for incoming requests. I was recommended to do a telnet test do debug and to use Sendgrid api keys to make the connection work better. So I removed the new port rules. Then I implemented the recommendations.
Telnet check
To do a telnet test you have to get a key and convert it to the appropriate version to do a test with it using telnet securely. So I went to Sendgrid, generated an api key with full access minus billing. Then I converted it to base64 with openssl from the command line using:
echo '<<YOUR_API_KEY>>' | openssl base64
I stored the api key and converted key in KeepassX for later use. When I just ran
telnet smtp.sendgrid.net 587
from the Trellis server in question I got:
telnet smtp.sendgrid.net 587 Trying 108.168.183.160... telnet: Unable to connect to remote host: Connection timed out
Well, and that was the error we had in the logs basically.
DO Ipv6 mail issues?
Then I read Digital Ocean’s port setup. So it seemed it was an ipv6 Digital Ocean port issue. So based on this DO question I edited gai.conf:
nano /etc/gai.confand made the appropriate lines look like this:
precedence ::ffff:0:0/96 100
where 10 becomes 100 and the whole line is uncommented. This to run via ipv4. Well, it did not help.
Sendgrid API Plugin
So I installed the Sendgrid API plugin. Adding details in Safari got the Sendgrid settings page reloading like crazy . In Chrome things did work fine as well as a test email using the plugins settings page for this.
Final Test with Sendgrid API
So final test that needed to be done was a new (test) purchase and see if Sendgrid was working and no longer blocking the whole checkout process. I did and the payment worked, a on page and by email confirmation were done right away. And that is amazing news. Sendgrid API all the way!
NB Did have one JS error in the console stil:
TypeError: undefined is not an object (evaluating '$(".woocommerce-billing-fields__field-wrapper").position().left')
but that may be caused by other plugins used on the page and did not seem to interfere. So that can be debugged in time.