Self Hosted - SMTP Emails - Gmail detecting as SPAM

[shanedel]

Hi have just installed a self hosted version of InvoiceNinja v4.5.10 and have to say its absolutely brilliant.

The only issue I can come across is that any invoices emailed to any gmail account are flagged as spam.
Emails are being sent via SMTP from my own domain.

Emails sent from this domain via outlook or webmail are not flagged as spam so I presume it is an issue with the template from invoice ninja or the way its being sent.

I have no issues receiving emails to hotmail.com, outlook.com, yahoo.com etc. it just seems to be gmail from what I have tested.

Has anyone else experienced this before and how did you overcome it?

[hillel]

Hmmm… not sure, you may want to see if changing the email template or attaching/not attaching the PDF has an impact. I’d also suggest testing with a different email provider.

[jmadrone]

Yes, I am experiencing this and have not found anything to solve this problem. Do you have proper authentication, DKIM SPF & DMARC setup for your domain? Have you tested this with something like http://mail-tester.com?

Here is my post: Invoice emails to clients marked spam and “Suspicious link” warning messages

[ds-systems]

Hi,

We have the same problem. SPF, DKIM, DMARC, all test passed. The problem is that, when the PDF is attached by phantomjs service, the mail is detected as potencialy dangerous. If the PDF is not attached, the mail is received in inbox.

Any advice to attach the PDF and going message to inbox?

Thanks a lot.

[encounters]

I’m having this exact similar problem. Can you guys please advise? I’ve done everything to get this working.

[semotech]

There should be settings in Gmail to whitelist sender emails and IP’s. You can add both the server IP and the sending email in the advanced configuration for Gmail and that should solve it.

[encounters]

the problem is on the receivers side

[metainnova]

Hi @encounters,

In our case, the problem was in the HTML design. The source code had a lot of references to other servers (like logo in mailchimp servers), external css, etc. When we edited the source code, we uploaded all css and images to invoiceninja ftp, changed url referrals, and after that, tara! notifications in inbox including at Microsoft Hotmail / Outlook 365 mail inboxes.

Review your mail HTML code and fix it.

We hope this will help you.

Good Luck!

[foliovision]

@metainnova

Thanks for your notes:

In our case, the problem was in the HTML design. The source code had a lot of references to other servers (like logo in mailchimp servers), external css, etc. When we edited the source code, we uploaded all css and images to invoiceninja ftp, changed url referrals, and after that, tara! notifications in inbox including at Microsoft Hotmail / Outlook 365 mail inboxes.

We are also running InvoiceNinja self-hosted (via Cloudron on DigitalOcean, the install is very easy with all DNS, SPF, DKIM, DMARC set up properly and accurately: we know how to do all of that properly but it takes us much longer to handroll and test).

With White Label all of those issues go away with no external CSS or images in the emails sent to clients. Emails to clients are the ones which worry me the most. We can troubleshoot our own email server. Here’s what our spam assassin has to say about white label emails:

Content analysis details:   (-1.8 points, 5.0 required)
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
                             [score: 0.0001]
  0.0 T_SPF_PERMERROR        SPF: test of record failed (permerror)
  0.0 T_OBFU_PDF_ATTACH      BODY: PDF attachment with generic MIME type
  0.3 HTML_MESSAGE           BODY: HTML included in message
  0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily
                             valid
 -0.1 DKIM_VALID_AU          Message has a valid DKIM or DK signature from
                             author's domain
 -0.1 DKIM_VALID             Message has at least one valid DKIM or DK signature
 -0.1 DKIM_VALID_EF          Message has a valid DKIM or DK signature from
                             envelope-from domain

Those familiar with SpamAssassin will recognise that a negative score is excellent. It means that the email would almost never be mistaken for spam.

White label cost is $30/year, the minimum I would contribute to an open-source project we use in production). I’m delighted to pay it.

On the other hand, I’m none too thrilled that the backend and the internal emails are both ugly and have way too much InvoiceNinja branding.

[foliovision]

I should add that the backend is not ugly but cannot have its colours or branding customised, while the outgoing internal emails are in fact ugly and do have far too much InvoiceNinja branding everywhere.

White label should be more or less white label everywhere. If the backend and the internal emails keep a small mention of InvoiceNinja that’s fine but the branding is too heavy-handed. The internal emails particularly bother me.

[metainnova]

Hi @foliovision

Thanks for the information. We have two different problems to solve here.

First of all, in the headers of that message figures an SPF problem (SPF: test of record failed (permerror)). You should fix that at first. We recommend you to make a test message to a GMAIL/GSuite account, and view complete headers. You should view “pass” in the three “filter” tests (SPF, DKIM and DMARC). When you test and have all in “green”, go to next step.

Complementary to first step, you need to verify the reputation of your mail server. Google uses mainly dnsbl blacklist. Check you don’t have your server in any of them. Having your server blacklisted causes mails goes to SPAM folder, greylist, and blacklists. You can use https://www.dnsbl.info

Check the reputation punctuation at https://www.senderscore.org/lookup.php?lookup=0. Microsoft uses that system to evaluate receiving the mail in SPAM or in Inbox. If your server has a good reputation, perfect. If not, you could use a mail provider / server with good score. Our provider has 98% excellent score.

At last but not least, you should take a look to the content of the mail. Invoiceninja uses templates for HTML. We recommend you to make all test with Microsoft, because if mails goes to Inbox in Microsoft Hotmail Service, in all the others will go to Inbox. Microsoft is the worst provider to deliver mail. Send test mail with invoices (send invoice By mail) to that account (configure it in any client). Use firstly plain text message (no html). If client will receive ok, go to HTML notifications. If not, you have a problem in early steps to solve.

When the plan message goes to Inbox, use HTML. In that case, you can enter the content into Invoice ninja (but the format is very limited), or made a custom HTML modifying the HTML template directly into the source code (we do that). And check, and check…

In our case, making a good design, responsive, through the html template in the source code, make us the best. A complete personalized message, and always goes to Inbox.

We hope this message could help you.

Good Luck!

[foliovision]

@metainnova I don’t seem to have been clear enough here. Just adding White Label to Cloudron and using plain text email notifications (the HTML ones are pretty ugly) will get you very clean emails which should not trigger spam flags.

The code block above is not email headers but Spam Assassin ratings for the incoming email after passing through the incoming email server.

SPF record: our DNS, SPF, DKIM and DMARC all check out green in Cloudron. Automatically. DigitalOcean DNS integration with Cloudron works perfectly.

All this manual work you are asking people to do is basically unnecessary. Just subscribe to the White Label option (goes directly to support the InvoiceNinja project and its developer Hillel Coren).

To my mind, Cloudron is about working smarter not harder.

[metainnova]

Hi @foliovision

We don’t understand your message. We don’t have any problem with our mails in our invoiceninja setup. We had a problem one year ago with considering HTML messages as SPAM and we solved it like we have said months ago. Our problem was with Hotmail clients email accounts, for having external references in our HTML design, and we fixed it as we have said.

We don’t use Cloudtron. They are a lot of different providers (OVH, AWS, etc…) and in our case, we have our own physical servers and infrastructure. We are a hosting company like Cloudtron, and we are experts in mail servers and hosting solutions in Spain.

Paying white label is not relevant to having the warranty your messages are going to deliver into inbox. Appart of that, if using cloudtron, and sending in txt format, is enough for you, and receive your client the messages in inbox, perfect. We don’t need to say anything else. If you think your messages are going to SPAM, like CEO of a hosting company, Metainnova, we have to say you the problem is not with your invoice ninja setup, white label, etc., is with the IP range of your VPS provider. Surelly, your IP is blacklisted in any DNSBL list, and is irrelevant you have in your provider SPF, DKIM or DMARC in white. They exists more filters, like DNSBL and SenderScore (the punctuation of the server is sending the emails.

Sorry for insisting you, but we are experts in this area, and is important checking the delivery of the message in the destination (not in your side, the senders side). So please, if you want to verify why the destination is sending the mail to SPAM, you have to take a look to the punctuation of the IP server is sending the message (contact your provider if you don’t know what is the IP of the mail service, or verify the IP of your VPS if is a dedicated VPS and sent directly for it), and verify there is not in any DNSBL. The webpages we have sent to you are used by Microsoft and Google.

In the other side, I disagree with you with HTML design posibilities. Efectively, the design of default HTML messages in invoiceninja is simple or ugly, but if you have HTML5 and CSS knowledge, you can create beautiful and professional email notifications modifying source code (and of course, we are agree with you with collaborating with the project and with Hillel Coren).

We hope to have been helpful.

[foliovision]

@metainnova

The service is called Cloudron not Cloudtron. In technology, spelling is very important. There’s a big difference between r and rm.

For others who are buried in the very long, pseudo-expert and slightly off-topic posts of metainnova, there’s abolutely no need to touch the source files to clean up outbound invoices to pass spam filtering. Just purchase and turn on White Label and use the plain text option (I haven’t tested the HTML versions after white labelling, they may be okay).

Of course as with any email it’s essential to set up SPF, DKIM and even DMARC properly these days for deliverability via Microsoft or Gmail servers.

[metainnova]

Good afternoon @foliovision

Honestly, it seems that more than asking for advice and sharing experiences through this chat, you are part of the commercial sales department of Cloudron (my apologies for spelling mistake) and Invoice Ninja.

I’m not going to go into your game about who knows more. You have your company and I have mine. You do things one way, I do them another. I have no interest in measuring the size of our ego.

We are licensed White Label, we regularly deploy Invoice Ninja as software as a service to our clients, we have established conversations with the product developers to develop a branch in their code to implement regulations that affect Spain and the Basque Country such as TicketBAI (and the treatment has been exceptional) and it is not possible to customize the HTML in conditions with the options that Invoice Ninja currently gives. If that’s enough for you, and you are in love with TXT notifications, great. In our case, we have made a responsive template in the style of the telephone and electricity companies, giving a very good image in the design, and that’s only possible at source-code level.

About the rest, I will not comment on anything else, because more than following or evaluating the advice from expert to expert (as we have considered you), we have received nothing but criticism and contempt.

We are not going to answer anything else on this topic.

Sadly, with members of your kind, we don’t want to collaborate. We collaborate with a lot of different software develops, products and developer communities, and we have never encountered behavior like yours.

Sincerely,

Daniel M. (CEO, Metainnova Solutions)

[metainnova]

By the way, some examples of customization and HTML template that can be done with programming knowledge, and always complying with the licensing terms for individual use and not marketing.

[varrick]

Seems this convo is going of topic - I do not understand how the mail is routed @FOLIOVISION. Is it running from Cloudron direct to customer or through your domain host? I suspect mail routing may be the problem here (assumption). Since you mentioned in the beginning webmail is fine but from invoice ninja it is not.

Have to be honest been running invoice ninja for years - html links or format have never been a problem not to sure how that became part of the discussion either - however we do not not use phantomjs cloud but rather local. Lets stay on point and get you going if you are still struggling.