Problems migrating from v4.5.50 to v5.7.7

I’m trying to migrate my 4.5.50 installation to 5.7.7. The migration appears to complete successfully, but there are some problems with the data in the new installation. I receive the “Migration Completed” mail, which shows errors. The same errors show up when I run “php artisan ninja:check-data”:

2023-09-04 11:38:51 2023-09-04 11:38:51 Running CheckData... on Connected to Default DB Fix Status = Just checking issues
2023-09-04 11:38:51 0 clients with incorrect balances
2023-09-04 11:38:52 0 clients with incorrect paid to dates
2023-09-04 11:38:52 0 contacts without a contact_key
2023-09-04 11:38:52 0 clients without any contacts
2023-09-04 11:38:52 0 contacts without a contact_key
2023-09-04 11:38:52 0 vendors without any contacts
2023-09-04 11:38:52 88 activities records with incorrect invoice company id
2023-09-04 11:38:52 88 activities records with incorrect payment company id
2023-09-04 11:38:52 88 invoices records with incorrect client company id
2023-09-04 11:38:52 88 payments records with incorrect client company id
2023-09-04 11:38:52 0 wrong invoices with bad balance state
2023-09-04 11:38:52 0 Contacts with Send Email = true but no email address
2023-09-04 11:38:52 0 users with duplicate oauth ids
2023-09-04 11:38:52 Done: FAILURE
2023-09-04 11:38:52 Total execution time in seconds: 0.1864218711853

Don’t see any errors in either laravel.log.

The v5 database contains all 88 invoices from v4, but the web interface only shows 39 of them. Similar problem with the 259 expenses in v4, the v5 database contains 260 (so 1 bogus?), but the web interface only shows 159. Similar with projects, the v5 db contains all 2 that exist in v4, web shows only 1. Tasks db in v5 has all 22 tasks from v4, web only displays 8.

Clients, products, vendors appear to have been properly migrated.

HI,

I believe updating to 5.7.8 should correct this.

Thanks for your response!

Unfortunately, after updating my v5 instance to v5.7.8, wiping the database completely, running setup again, and finally doing the migration again, the behaviour is exactly the same.

@david can you please advise?

I mean at this point it would be easier to export data, spin up a fresh instance (separate) and import data with verification.
Kill the old instance after some testing period.

Not sure what you mean with “import data with verification”. Also, the export feature does not export expenses nor tasks, both of which I would very much like to keep…

1 Like

We’ve tagged a new release (v5.7.9) with a fix for the check data issue.

To make sure the problem doesn’t already exist in the v4 instance, I ran check-data there also:

2023-09-06 10:30:36 2023-09-06 10:30:36 Running CheckData...
2023-09-06 10:30:36 0 contacts without a contact_key
2023-09-06 10:30:36 0 clients without any contacts
2023-09-06 10:30:36 0 clients without a single primary contact
2023-09-06 10:30:36 0 activities with blank invoice backup
2023-09-06 10:30:36 0 clients with incorrect paid to date
2023-09-06 10:30:36 0 draft sent invoices
2023-09-06 10:30:36 0 invoices with incorrect balances
2023-09-06 10:30:36 0 clients with incorrect balance/activities
2023-09-06 10:30:36 0 user accounts with multiple companies
2023-09-06 10:30:36 0 users with duplicate oauth ids
2023-09-06 10:30:36 0 failed jobs
2023-09-06 10:30:36 Done: SUCCESS

Updated to 5.7.9, wiped database, ran setup, did migration again and unfortunately again the exact same behaviour.

by verification I meant just to verify it imported correctly.
that is a bit concerning - not being able to export 100% of data. Thank for letting me know.

@hillel - any plans for this?

5.7.9 should fix the check data script, we’ll correct the sorting in an upcoming release of the Flutter app.

All data should be migrated, you can compare the dashboard totals to check for accuracy.

So it seems that with 5.7.9, the check data script returns no errors:

2023-09-16 01:01:12 2023-09-16 01:01:12 Running CheckData... on Connected to Default DB Fix Status = Just checking issues
2023-09-16 01:01:12 0 clients with incorrect balances
2023-09-16 01:01:12 0 clients with incorrect paid to dates
2023-09-16 01:01:12 0 contacts without a contact_key
2023-09-16 01:01:12 0 clients without any contacts
2023-09-16 01:01:12 0 contacts without a contact_key
2023-09-16 01:01:12 0 vendors without any contacts
2023-09-16 01:01:12 0 wrong invoices with bad balance state
2023-09-16 01:01:12 0 Contacts with Send Email = true but no email address
2023-09-16 01:01:12 0 users with duplicate oauth ids
2023-09-16 01:01:12 Done: SUCCESS
2023-09-16 01:01:12 Total execution time in seconds: 0.14844608306885

However, in the web UI, there are still missing invoices, payments, expenses. Like before, they are all there in the database, but not everything that is in the database shows up in the UI for some reason.

Have you tried both the Flutter and React web interfaces?

If you have a large account the Flutter app will only load recent data by default, you can change the number of years loaded on Settings > User Details.

Ahhh, I totally missed that setting. It was set to 3 years and I have 8 years worth of data in my instance. With that, I can conclude my migration, archive my v4 instance and ditch PHP 7.

Thanks a lot!

Glad to hear it, thanks for the update!