Error during forced Migration v4 -> v5

ecomsilio · January 12, 2021, 10:36pm

Hi,

the migration from v4 to v5 doesn’t do anything in my case. I don’t see any errors in v4 or v5.

I have found a zip file at /var/www/invoiceninja/public/storage/migrations/
I then tried to run the migration process manually with php artisan migrations:import but that throws the error:

DirectoryIterator::__construct(/var/www/invoiceninja/public/storage/migrations/import): failed to open dir: No such file or directory

My .env file says QUEUE_CONNECTION=database. Because it is not empty, I am afraid I would break it by changing it so QUEUE_CONNECTION=sync.
I then went back to v4 and forced the migration. This resulted in the following error in the laravel.log

Error:

[2021-01-12 22:23:35] production.INFO: Importing account  

[2021-01-12 22:23:35] production.INFO: Importing company  

[2021-01-12 22:23:35] production.INFO: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'logo' in 'field list' (SQL: update `companies` set `industry_id` = 9, `ip` = 88.153.232.56, `company_key` = y33feijmdzo41u51a4dlp61cpphdngfy, `update_products` = 0, `size_id` = 2, `enable_modules` = 31, `cus.....

What can I do?

Thanks,
Gijs

ben · January 13, 2021, 9:00am

Hey there, thanks for reporting this.

@david do you know anything about this issue?

david · January 13, 2021, 9:15am

Hi @ecomsilio

this is a bug, will be fixed in next v4 release.

ecomsilio · January 13, 2021, 10:13am

Hi @david

Perfect. Is there anything I need to do to run the migrations a second time?

Thanks,
Gijs

david · January 13, 2021, 10:25am

no, nothing migration related, this is just a small code change.

Ubi · January 14, 2021, 9:14am

Hi guys,
exactly the same error log trying to migrate from V4.5.28 -> 5.0.47.
I am wondering if is not easier just to export clients, products and invoices from V4 and then import clients, products and invoices to V5 manually via GUI.
What is the difference beside to setup company manually before importing CSVs?

Thanks

ben · January 14, 2021, 9:17am

Hey @Ubi 4.5.29 has been released few hours ago, can you please update?

david · January 14, 2021, 9:17am

hi @Ubi please update to v4.5.29

Ubi · January 14, 2021, 10:37am

Thank you for your prompt reply. It seems that it’s getting better because i don’t see any new log on laravel.ol file but nothing is imported. The only “new” thing i can see all the time i try tio run the migration tool is the new company record from the top-left dropdown but everything is empty.

Ubi · January 14, 2021, 10:40am

Do i have to use the same license between V4 and V5 perhaps? V.4 is the licensed one

ecomsilio · January 14, 2021, 11:05am

I have updated to v4.5.29 and v5.0.47-C38 and I have tried the migration again. I now get the following error.

v5

[2021-01-14 10:56:36] production.INFO: Importing invoices  
[2021-01-14 10:56:36] production.INFO: Missing resource key: client_contacts_5  
[2021-01-14 10:56:37] production.ERROR: Missing resource key: client_contacts_5 {"userId":1,"exception":"[object] (Exception(code: 0): Missing resource key: client_contacts_5 at /var/www/invoiceninja/app/Jobs/Util/Import.php:1324)
[stacktrace]

v4

[2020-09-14 20:09:55] production.ERROR: ***InvalidArgumentException*** [0] : /var/www/html/invoiceninja/vendor/laravel/framework/src/Illuminate/Support/Manager.php [Line 91] => Driver [.env] not supported. {"context":"PHP","user_id":0,"account_id":0,"user_name":"","method":"GET","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0","locale":"en","ip":"185.234.218.42","count":1,"is_console":"no","is_api":"no","db_server":"mysql","url":"auth/.env"} []

ecomsilio · January 14, 2021, 1:16pm

Because of the “client_contacts_5” in the error message, I decided to delete client with ID 5 including all connected invoices etc. (Made a snapshot first to be able to go back again).

I removed the migration files from the v5 server and used php artisan migrate:fresh --seedto get a fresh setup. I then went through the migration process again. Still the same error.

I then looked up the contact with ID 5 instead of the client with ID 5 and was able to find it using the REST Call GET /api/v1/contacts/5
The contact was not connected to a client anymore. When I tried to delete the contact using DEL /api/v1/contacts/5 (not sure if this route exists) and I got a 302 Response redirecting me to my InvoiceNinja URL.

I then tried to update an existing client using PUT /api/v1/clients/[Client ID] which gave me a 404 Not Found response saying “record does not exist”.

Payload:

{
  "account_key": "y.....gfy",
  "contacts": [
    {
      "account_key": "y.....gfy",
      "is_owner": true,
      "id": 40,
      "first_name": "John",
      "last_name": "Smith",
      "email": "john@smith.com",
      "contact_key": "b.....prb",
      "updated_at": 1581880342,
      "archived_at": null,
      "is_primary": true,
      "phone": "876543567890",
      "last_login": "",
      "send_invoice": true,
      "custom_value1": "BA9876543567",
      "custom_value2": "12BA"
    },
    {
      "account_key": "y.....gfy",
      "is_owner": true,
      "id": 5,
      "first_name": "Max",
      "last_name": "Mustermann",
      "email": "max@wefef.com",
      "contact_key": "a.....bsh",
      "updated_at": 1581880342,
      "archived_at": null,
      "is_primary": false,
      "phone": "87654345",
      "last_login": "",
      "send_invoice": true,
      "custom_value1": "AB3456789",
      "custom_value2": "AB12344"
    }
  ]
}

I am not sure why this client.contact ID 5 is causing this error.
Anything I can do to get this migration to a success?

Thanks,
Gijs

david · January 14, 2021, 10:57pm

Hi Gijs,

This is a new one for me.

To start debugging this, I would first confirm that the contact was actually migrated to V5. Then we’ll be able to debug further.

Ubi · January 15, 2021, 7:40am

Hi @david, just to confirm that i let it run the migration all night long and at ceratain point it starts. I don’t know how and why it took some time but this morning i wake up with the company correctly imported. I just not get the final email as written like confirmation on V4 migration tool.
So, i chek the logs and i can see :

[2021-01-14 00:01:02] production.INFO: latest version = 5.0.46  
[2021-01-14 00:01:02] production.INFO: updating currencies  
[2021-01-15 03:51:02] production.INFO: Importing account  
[2021-01-15 03:51:02] production.INFO: Importing company  
[2021-01-15 03:51:02] production.INFO: the path /storage/5jynr....K9.png  
[2021-01-15 03:51:02] production.INFO: Importing users  
[2021-01-15 03:51:02] production.INFO: Importing payment_terms  
[2021-01-15 03:51:02] production.INFO: Importing tax_rates  
[2021-01-15 03:51:02] production.INFO: Importing clients  
[2021-01-15 03:51:07] production.INFO: Importing company_gateways  
[2021-01-15 03:51:07] production.INFO: Importing client_gateway_tokens  
[2021-01-15 03:51:07] production.INFO: Importing vendors  
[2021-01-15 03:51:07] production.INFO: Importing projects  
[2021-01-15 03:51:07] production.INFO: Importing products  
[2021-01-15 03:51:07] production.INFO: Importing credits  
[2021-01-15 03:51:07] production.INFO: Importing invoices  
[2021-01-15 03:51:12] production.INFO: Importing recurring_invoices  
[2021-01-15 03:51:12] production.INFO: Importing quotes  
[2021-01-15 03:51:13] production.INFO: Importing payments  
[2021-01-15 03:51:18] production.INFO: Importing expense_categories  
[2021-01-15 03:51:18] production.INFO: Importing task_statuses  
[2021-01-15 03:51:18] production.INFO: in task statuses  
[2021-01-15 03:51:18] production.INFO: finished task statuses  
[2021-01-15 03:51:18] production.INFO: Importing expenses  
[2021-01-15 03:51:18] production.INFO: Importing tasks  
[2021-01-15 03:51:18] production.INFO: Importing documents  
[2021-01-15 03:51:18] production.INFO: Address in mailbox given [] does not comply with RFC 2822, 3.6.2.  
[2021-01-15 03:51:19] production.ERROR: Address in mailbox given [] does not comply with RFC 2822, 3.6.2. {"userId":1,"exception":"[object] (Swift_RfcComplianceException(code: 0): Address in mailbox given [] does not comply with RFC 2822, 3.6.2. at /home/.../public_html/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Mime/Headers/MailboxHeader.php:355)

And this is explaining why i didn’t get the email

SO very happily i start to check how to re-set invoices layout and flavour, but when i press on V5 “customoize & preview” i got a 500 error modal pop up without deatils, just ERROR 500 :

I check the logs again and this is the outcome

[2021-01-15 03:51:19] production.INFO: latest version = 5.0.47  
[2021-01-15 03:51:19] production.INFO: updating currencies  
[2021-01-15 06:41:03] production.ERROR: Downloaded Chromium binary is not executable. Make sure to set correct permissions (0755) {"userId":1,"exception":"[object] (Beganovich\\Snappdf\\Exception\\BinaryNotExecutable(code: 0): Downloaded Chromium binary is not executable. Make sure to set correct permissions (0755) at /home/......mmm//public_html/vendor/beganovich/snappdf/src/Snappdf.php:57)

Thank you

david · January 15, 2021, 8:54am

So this is a easy one, you need to chmod +x the chromium binary

@ben can you help with the location of the binary?

ecomsilio · January 15, 2021, 9:03am

Hi @david

How do I find this contact in v5?
In v4 the contact is not searchable and is not connected anymore to a client. I only can find it via the REST API (GET /api/v1/contacts/5).

How do I check for this contact in v5 via the API? I have found a route for GET /api/v1/clients which contains the contacts and a route for GET /api/v1/companies but that doesn’t contain clients or contacts. I can’t find a route specific for contacts.

Thanks,
Gijs

@Ubi
I had the same error and this helped me:

and

Ubi · January 15, 2021, 9:51am

Hi @ecomsilio, thank you, i tried but still… same error. The difference there is that I am on Debian 10, my library is named just chromium, not chromium.brower and I am not sure i did everything correctly.

Saiyng that, yes david and @ben, where is located the library ? I have this path once installed :

Thank you again guys.

ben · January 15, 2021, 10:21am

If you use vendor/bin/snappdf download, the binary will be located in vendor/beganovich/snappdf/versions. However, it is auto-wired for you so no need for additional actions.

Ubi · January 15, 2021, 10:31am

Thank you @ben ! It worked right after i ran

and no new rows on laravel.log

Seems all ok now, awesome.

ben · January 15, 2021, 10:32am

Great, happy you got it working. It’s weird cause chmod() is part of the download script so it should do it for you.