V4 to v5 migration failed: invoice/quote document not available

Hi,

I have set up a v5 instance of InvoiceNinja on Docker using docker-compose.
When I am trying to do the migration via the v4 UI I am getting the following error:

Looks like something went wrong with the migration for the following company: XXXX

Resource invoice/quote document not available.
Resource invoice/quote document not available.

Can someone help me what this means? I was looking at the laravel*.logs but I could not find anything.

Hi,

Are you using the latest versions of v4 and v5?

I am using v4.5.42 and v5.2.15-C56 (Docker installation)

Update: I have upgraded to v4.5.43 but still with the same result :frowning:

The latest v4 is v4.5.43, it may help.

@david any thoughts?

My update come parallel to your reply :slight_smile:
I did update the v4 version but it is not working :frowning:

Any way I can give more information? E.g. where can I find more detailed logs?

The logs are in the storage/logs/ folder.

Thanks. I have started another migration and tailed -f the stacktrace.log and the laravel-error.log. Unfortunately I did not get any new entry. Any other option to debug this more without looking at the source code? Or is there anything else I can provide you so that you can give me some pointers?

@terencejackson8000

This type of failure indicates the migration couldn’t find the reference entity that the document belongs to:

I believe in the latest version of v4 and v5, we can jump over these failure so they don’t prevent the entire migration from falling over.

If you don’t need all the documents to be migrated, then all your data except those documents will have been transferred to your v5

@david Thanks for your response.

It looks like something has been though
[2021-08-10 07:29:15] production.INFO: Importing account
[2021-08-10 07:29:15] production.INFO: Importing company
[2021-08-10 07:29:16] production.INFO: Importing users
[2021-08-10 07:29:16] production.INFO: Importing payment_terms
[2021-08-10 07:29:16] production.INFO: Importing tax_rates
[2021-08-10 07:29:16] production.INFO: Importing clients
[2021-08-10 07:29:29] production.INFO: Importing company_gateways
[2021-08-10 07:29:29] production.INFO: Importing client_gateway_tokens
[2021-08-10 07:29:29] production.INFO: Importing vendors
[2021-08-10 07:29:29] production.INFO: Importing projects
[2021-08-10 07:29:29] production.INFO: Importing products
[2021-08-10 07:29:31] production.INFO: Importing credits
[2021-08-10 07:29:31] production.INFO: Importing invoices
[2021-08-10 07:31:05] production.INFO: Importing recurring_invoices
[2021-08-10 07:31:10] production.INFO: Importing quotes
[2021-08-10 07:31:18] production.INFO: Importing payments
[2021-08-10 07:32:43] production.INFO: Importing expense_categories
[2021-08-10 07:32:43] production.INFO: Importing task_statuses
[2021-08-10 07:32:43] production.INFO: in task statuses
[2021-08-10 07:32:43] production.INFO: finished task statuses
[2021-08-10 07:32:43] production.INFO: Importing expenses
[2021-08-10 07:32:43] production.INFO: Importing tasks
[2021-08-10 07:32:43] production.INFO: Importing documents

And I also see data in the database:

I do not see anything on the UI though. How can I solve this?
I have restarted the docker to make sure this is not a caching issue, but the issue is not fixed with this.

Do I need to update the data manually in some shape or form? (E.g. assign it to a company or so?)

Do you see more than one company in the top left company selector? It may help to change companies.

I only see one company
@hillel Please also check my edit. I was a little too fast in my previous response :slight_smile:

After startng another migration it seem that on the database everything looks good: New company, all invoices linked to the company, users are linked to the company.

Unfortuntely I am now running into a timeout issue when logging in:

TimeoutException after 0:00:30.000000: Future not completed

Sometimes I am also getting a XMLHttpRequest error.

In the logs I am seeing a warning:
2021/08/10 20:31:17 [warn] 24#24: *78 an upstream response is buffered to a temporary file /var/cache/nginx/fastcgi_temp/8/00/0000000008 while reading upstream, client: 192.168.x.x, server: _, request: “POST /api/v1/login?first_load=true&include_static=true HTTP/1.1”, upstream: “fastcgi://172.22.0.3:9000”, host: “192.168.x.x:xx”, referrer: “https://ninja.xxx.xx/
192.168.2.4 - - [10/Aug/2021:20:31:17 +0000] “POST /api/v1/login?first_load=true&include_static=true HTTP/1.1” 200 548790 “https://ninja.xxx.xx/” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36” “195.76.177.125”

Any idea why I am suddenly running into a timeout? Migrated data is roughly 2k invoices.

@terencejackson8000

How large is your dataset?

If it is very large, you may need to set the column is_large to 1 so that the front end chunks the data rather than attempting to load it all in one go.

The dataset actually is not that large, ~3MB in total:

Nevermind, I found it. It is in the companies table:
UPDATE companiesSETis_large= '1' WHEREcompanies.id = 1;

This worked, thanks a lot. I can see the new company now and also see the data.

Here are few findings from my side:

  • If the company you are migrating has the same name as a company that already exists, it did not work for me => I then changed the name of the initial company to something different
  • Change the is_large to 1 in the companies table (Thanks @david)
  • Logout and Login after the migration. A simple refresh does not show the data

@david I thought the issue was solved but now it still happens from time to time. Sometimes I am able to login, sometimes I am not.

What else could be the issue here? And are there other ways to optimize this?

I’ve also seen similiar, if two company names are the same, only one appears in the drop down, cc @hillel

Are you running on shared hosting? It may be an underpowered server if the issue is intermittent.

Thanks for the input @david
I am running this on a local machine, no shared hosting (8GB ram, 2GHz CPU, 2 cores). CUP and RAM usage is not very high though.

@terencejackson8000

I think we are spread out now over 2 threads. Am i correct to say that this issue is resolved, and now the focus is on the migrated custom_fields being the issue?

Correct, the other thread is closed. This is for the custom_fields issue. I figured it make sense to open a new thread for this