Invoice emails to clients marked spam and "Suspicious link" warning messages

Self-Hosted | v4/Legacy
1

Environment:

  • Self-hosted
  • v4.5.10

I am having a heck of a time with every email sent to real and test clients being marked as spam. Even worse, even after marking a message as ‘Not spam’ in Gmail, clicking the ‘Pay’ button in message body results in a popup window with warning message that says:

Suspicious link This link leads to an untrusted site. Are you sure you want to proceed to invoice.mydomain.com?

I am using a subdomain for my self-host install (eg. invoice.mydomain.com) and Gsuite (Google) SMTP servers to send mail. I have a user setup in Gsuite [email protected] that is used to send mail, so from and reply-to are the same.

My domain has valid authentication with SPF, DKIM, and DMARC and receive a 9/10 score from mail-tester.com (the -1 point is for redirection found and url not formatted properly). I am using standard email templates with no additional html added.

I have spent many hours reading many different topics about this and believe to have implemented every suggested fix and still no resolution. I don’t have any other issues with other emails whatsoever.

Has anyone found a successful fix to this problem?

2

Do you have https setup on your site? gmail throwing the error

This link leads to an untrusted site. Are you sure you want to proceed to invoice.mydomain.com?

would indicate the site is not secured by SSL, or the link is not being generated to the correct https:// link

In regards to email delivery, this is not an issue with Invoice Ninja, this will be the way you have your email server configured there will be clues in the headers on the received email as to why it was marked as spam.

3

Hi and thanks for the quick reply.

Yes I do have a valid cert and https is setup correctly. I understand that Invoice Ninja is just using the SMTP server that I define to send the messages. I do have one development since my post…

Completely removing the email signature in settings, which did have a link to the client portal that was misspelled, has solved the suspicious link messages. I think that’s precisely what you were referring to in your message to me.

I’m not sure exactly what’s causing the spam messages, but I have eliminated everything but the default templates with no additional text to see if that helps. I am sending test invoices to multiple email addresses I own and see that even within Gmail I get different results, depending on the security settings that are configured within Gsuite Admin Control Panel (not sure if Gmail users can set these). On one domain I have much looser settings, specifically I have turned off the two that relate to ‘Phishing’ emails, and invoices are coming to that inbox no problem. In another domain, with all security options enabled, the messages are identified specifically as “being like other phishing messages”.

I will continue to troubleshoot tomorrow…

4

You may want to test with a different email provider and/or domain.

5

@jmadrone

have a look in the headers of a rejected email, all the information you need to debug this will be annotated in there.

6

DavidBomba can you tell me what I might be looking for in my email headers? I have since removed 100% of all links in my Email Signature and am using default templates with absolutely no additional anything. I am sending test emails to multiple addresses and providers and gmail is still being a real problem. Every invoice I send gets marked with as spam with suspected phishing warnings all over it and a red banner. That Suspicious Link warning message pops up no matter what you’re clicking on, so clicking the View Invoice button or even the domain.com link in the very bottom of the email that is not something I added but I’m guessing comes from my company information? I have downloaded the headers from several invoice emails and analyzed them with online tools as well as myself and don’t see anything out of the ordinary. I have checked my SSL cert with multiple online tools, which all checks out. I have used mxtollbox, mail-tester.com, Gsuite postmaster tools, etc and my DKIM, SPF, and DMARC all pass and pass acurately, meaning I have looked at each one and checked against what it should be. Thanks.

7

@jmadrone

What is your hosting platform?

Are you hosting using a shared hosting platform, or are you running your own virtual machine?

If the latter, can you confirm you have setup your hostname correctly on your machine, and also configured rdns to ensure google is not flagging your server as suspicious

8

I want to thank you for your quick response and continuing to assist with troubleshooting this. To answer your questions:

  1. Server is hosted with AWS EC2. IN is the only service running on this server... Nginx, MariaDB, and PHP 7.2
  2. Hostname = invoiceninja.mydoaim.com and was configured like this sudo hostnamectl set-hostname invoiceninja.mydomain.com
  3. rDNS = no, to be honest I don't know much about rDNS except for what I've read in the previous 5 mins. Correct me if I'm wrong, but wouldn't this be an issue if I was using this server as a mail server? Which it is not, as it is only using SMTP to send through Google's mail servers.
Thanks again
9

@jmadrone

From my understanding, even thou you are using gmail to send the mail, gmail will still look at the servers reputation that you are sending from, and it does mark down a server that may not be configured appropriately. As good practice I would always configure rDNS on any server.

If this fails, let us know, you’ll probably need to send through the email headers for us to look into the issue further.

10

After reading the email headers multiple times and not seeing anything that jumps out I ran a search for the word spam and found this:

X-CLX-Spam: false
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,
definitions=2019-02-27_14:, signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=54
malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=214 mlxscore=0
mlxlogscore=964 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
engine=8.0.1-1812120000 definitions=main-1902270139

I would be happy to pass along the headers, but they do have email addresses in them, so should I sanitize them and upload here, or send to you in an email, or something else?

11

Also interesting is that if I copy and paste all of the html in the email body of a message sent from IN server (AWS EC2 instance) that was marked as SPAM with all links contained therein SUSPICIOUS and send the same message from an the same email address via the same SMTP server using an email client (AirMail) the message is not marked SPAM and the links are not SUSPICIOUS.

I have also discovered that the parts of the message header above are related to Apple iCloud email and Proofpoint MX machine learning spam filtering software that they must use as those parts of the header are only in test messages to an icloud email address. I also believe they have to do with the fact that I have email forwarding setup on that email account to automatically forward email sent there to my Gmail account.

Sanitizing the full headers seems daunting, plus it makes it hard to follow/trace the various servers. Can I send to you directly or privately somehow?

12

It sounds like AWS is filtering outbound mail?

13

I have added my server’s IP address to my SPF record so that it now reads (sanitized) v=spf1 ip1.2.3.4 include:_spf.google.com ~all , although this is still somewhat confusing but my understanding is that this is not really necessary at all as my server is not sending the mail as it is merely connecting to Google’s SMTP servers. This assertion is supported by the fact that all email headers, SPF record tools, Mail-Tester.com, etc. - only ever show the IP address of the Google server that actually sent the mail. I don’t believe that it hurts anything to include it though and so I have.

I have also requested a PTR record for my AWS server using the form provided by Amazon online. This is a little different than the SPF record, but again my understanding is that my server (invoice.mydomain.com) is not actually sending the mail. Time to wait for this to propagate…

14

I think that is a logical place to look at this point DavidBomba. I am going to see if integrating Amazon SES helps in this matter. I have a Postmark account setup in Dev mode, but I really don’t want to spend the extra money at this point since I am already paying for multiple services such as Gsuite.

I am going to wait for the PTR to fully propagate and then implement SES and then report back.

15

I admit I did not read the whole thread but I was having a similar problem with my invoice being flagged as spam.

So here’s my advice: If you use Google SMPT don’t forget to include _spf.google.com in the SPF: https://support.google.com/a/answer/33786

Also don’t forget to install DMARC record it helped a lot for me: https://en.wikipedia.org/wiki/DMARC

Here’s the tools I used to troubleshot my issues:

    https://www.mail-tester.com/ Like you but it's only good 3 times a day in the free version. http://www.allaboutspam.com/email-server-test/ it's like the precedent but completely free and unlimited. http://www.isnotspam.com/ Another similar tool completely free https://mxtoolbox.com/SuperTool.aspx Here I use the free version and you have many tools to troubleshot everything I just used few of the tools.

I found your post and learned about https://www.mail-tester.com and it was the starting point for me to resolve my issues so it was helpful.

Regards :slight_smile:

16

Thanks @Mikhoul, that’s really helpful!

17

I have not found a solution to this problem. I have implemented SPF, DKIM, and DMARC, and have quadruple checked to ensure they are valid. I have a reverse DNS (PTR) record for my ec2 server so that the result of dig -x 1.2.3.4 returns something like this 4.3.2.1.in-addr.arpa. 299 IN PTR invoice.mydomain.com.,
although I don’t believe that is an issue because the headers are the same as without and do not reference ec2 server’s IP as “sending” mail server anywhere. I did add my IP to the SPF record as well which doesn’t change anything, again because all the messages still are sent from smtp.gmail.com. My SPF now reads v=spf1 ip4:1.2.3.4 include:_spf.google.com ~all.

As far as AWS being the culprit, I do not believe this to be the case because while they do filter port 25 and limit ability to send lots and lots of email from ports 465 & 587, I do not see anything in the headers to suggest any kind of machine learning/filtering being done, but I’m no expert in reading email headers so I could be wrong.

As of today my messages to Gmail accounts are still being marked as Dangerous Spam, with links disabled and a giant RED Banner at the top of screen that says:

This message seems dangerous Similar messages were used to steal people's personal information. Avoid clicking links, downloading attachments, or replying with personal information.

Then at the bottom of the screen, this message is displayed in orange banner:

Downloading these attachments is disabled. This email has been identified as phishing. If you want to download these and you trust this message, click "Not spam" in the banner above.

Google has several settings available to domain administrators which affect these messages. Depending on how the admin for the receiving domain has configured determines whether the messages make it through or not. The problem is that the default is “enabled” so I suspect most folks will have these “Safety” features on meaning they will not see my invoices/quotes/etc.

These settings can be found here:
Gsuite Admin > Apps > Settings for Gmail > Safety There are 3 settings:

  1. Attachments - Additional policies to protect against malware in emails.
  2. Links and external images - Additional settings to prevent email phishing due to links and external images.
  3. Spoofing and authentication - Additional settings to reduce phishing attacks due to spoofing and unauthenticated emails.
Each of the 3 settings above has multiple sub-levels and options too. I have been going through them to find out which specific one is the culprit, but it is time consuming and so far results are not definitive as they seem to change. The only thing that is consistent is the message content. The way gmail handles them varies.

I have been receiving DMARC reports from Postmark, which is great tool and feature they offer, and there does appear to be several “Unknown sources”, but there is nothing to be done about it according to Postmark’s FAQ and help guides.

I am currently waiting for Amazon Support to remove email sending restrictions from my server, as part of getting Amazon SES going. However, their support tells me that the only limits imposed are on Port 25. Period. Regardless, I made the request and am jumping the hoops.

How about that offer to look at some email headers?

18

It seems that the only way to prevent messages from going to spam is to contact each person using a personal email address you already communicate with them and request they look in their spam folder and add sender to their contacts, which doesn’t really seem like a good solution at all.

I suppose I could tell IN to send all emails using said personal email address, which would be much more likely to be in receivers contacts. I don’t recall ever receiving an electronic invoice from [email protected] though and it seems a bit unprofessional. Right now I am using a separate Gsuite user called [email protected]. This user is a full user setup in the Gsuite Admin control panel. Argh…

19

@jmadrone

Do your domain names align?

ie. The sending email user domain [email protected] is the same as the domain for the user embedded links? ie https://domain.com/view/invoice_url

are the embedded links using https?

20

@DavidBomba

I’m not sure I completely understand about the domain names aligning, but my setup is like this:

  • My company uses tld mydomain.com
  • IN uses subdomain invoice.mydomain.com
  • IN SMTP user= [email protected]

    • the [email protected] user is a full/regular user (ie. not an alias) so the envelope and header FROM address/user does match (ie. smtp username = [email protected] & Reply-to = [email protected]).

    As far as links go from what I can tell the emails consist of:

  1. logo image - http://mydomain.com (assuming this url is populated from the Company Details?)
    • I currently have mydomain.com in Company Details > Website
  2. $viewButton - url is `https://invoice.mydomain.com/view/ajfeopiahf;ioah;jvn;ioaeh’
  3. mydomain.com link in footer - again, assuming this is populated from Company Details > Website ??

I am using the Light Email Template

When I copy & paste all message content into a new message and send to the same 4 test client email addresses none of this happens (ie. Not marked Spam + links open with no warnings about untrusted sites).

Okay, so I would think that means that the content of the message is not causing problems. That makes me think AWS is filtering, but their support people have told me they are not, with the exception of port 25, which even that throttling has now been removed about an hour ago.