Best Way to migrate to V5

david · December 17, 2020, 9:20pm

It sounds like a permissions issue, just confirm /public/storage is writable by the web user.

gerootech · December 17, 2020, 9:22pm

@david it is
drwxrwxr-x 5 www-data www-data 4096 Dec 14 19:49 storage

Not sure where else to look to troubleshoot this issue.

This is the only error i can see now in the browser when i try to preview an invoice design:
a Popup in the broswer with 500 Internal server error
and in the console
T: http://in5.ghf.loc/api/v1/preview?
/api/v1/preview?:1 Failed to load resource: the server responded with a status of 500 (Internal Server Error)

When i try to view an generated invoice in PDF i get the following in the console:

main.dart.js?v=5.0.35:68120 GET http://in5.ghf.loc/client/invoice/0WdShGGrvbrHdgOTzmCx3fNqYMgd8tr4Dp41GfSbmH5fi8NsOpCODFcOqF6zWnrE/download_pdf 500 (Internal Server Error)

gerootech · December 20, 2020, 7:15am

@david,

I found the following errors while digging the laravel.log file more, seems npm is not able to launch chrome and npm is missing .config file on the /var/www path

I fixed the PDF by installing libgbm-dev, but i am not quite sure if i should be concerned about the npm error, i cannot find the .config file in the path mentioned above.

[2020-12-20 06:58:05] production.INFO: The command “PATH=$PATH:/usr/local/bin NODE_PATH=npm root -g node ‘/var/www/html/ninja5/vendor/spatie/browsershot/src/…/bin/browser.js’ '{“url”:“file:///tmp/289567636-0268609001608447481/index.html”,“action”:“pdf”,“options”:{“args”:[”–no-sandbox"],“viewport”:{“width”:800,“height”:600,“deviceScaleFactor”:1},“displayHeaderFooter”:false,“waitUntil”:“networkidle0”,“printBackground”:true}}’" failed.

Exit Code: 1(General error)

Working directory: /var/www/html/ninja5/public

Output:

Error Output:

┌──────────────────────────────────────────────────────┐
│ npm update check failed │
│ Try running with sudo or get access │
│ to the local update config store via │
│ sudo chown -R $USER:$(id -gn $USER) /var/www/.config │
└──────────────────────────────────────────────────────┘
Error: Failed to launch the browser process!
/var/www/html/ninja5/node_modules/puppeteer/.local-chromium/linux-800071/chrome-linux/chrome: error while loading shared libraries: libgbm.so.1: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

at onClose (/var/www/html/ninja5/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193:20)
at Interface.<anonymous> (/var/www/html/ninja5/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:183:68)
at Interface.emit (events.js:327:22)
at Interface.close (readline.js:424:8)
at Socket.onend (readline.js:202:10)
at Socket.emit (events.js:327:22)
at endReadableNT (internal/streams/readable.js:1327:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21)

[2020-12-20 06:58:06] production.ERROR: The file “/var/www/html/ninja5/public/storage/1w7yqbxctnznmz98dbqfz4impkxldkf5/p08jlJAgYT26buihAC7fdM07IBl8RA8QdUiu1VX5/invoices/INV-2020-33.pdf” does not exist {“exception”:"[object] (Symfony\Component\HttpFoundation\File\Exception\FileNotFoundException(code: 0): The file “/var/www/html/ninja5/public/storage/1w7yqbxctnznmz98dbqfz4impkxldkf5/p08jlJAgYT26buihAC7fdM07IBl8RA8QdUiu1VX5/invoices/INV-2020-33.pdf” does not exist at /var/www/html/ninja5/vendor/symfony/http-foundation/File/File.php:36)
[stacktrace]
#0 /var/www/html/ninja5/vendor/symfony/http-foundation/BinaryFileResponse.php(93): Symfony\Component\HttpFoundation\File\File->__construct(’/var/www/html/n…’)
#1 /var/www/html/ninja5/vendor/symfony/http-foundation/BinaryFileResponse.php(51): Symfony\Component\HttpFoundation\BinaryFileResponse->setFile(’/var/www/html/n…’, ‘attachment’, false, true)
#2 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/ResponseFactory.php(165): Symfony\Component\HttpFoundation\BinaryFileResponse->__construct(’/var/www/html/n…’, 200, Array, true, ‘attachment’)
#3 /var/www/html/ninja5/app/Http/Controllers/InvoiceController.php(797): Illuminate\Routing\ResponseFactory->download(’/var/www/html/n…’, ‘INV-2020-33.pdf’)
#4 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\Http\Controllers\InvoiceController->downloadPdf(‘jCrLXx78MK8M3JQ…’)
#5 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\Routing\Controller->callAction(‘downloadPdf’, Array)
#6 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Route.php(254): Illuminate\Routing\ControllerDispatcher->dispatch(Object(Illuminate\Routing\Route), Object(App\Http\Controllers\InvoiceController), ‘downloadPdf’)
#7 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Route.php(197): Illuminate\Routing\Route->runController()
#8 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Router.php(692): Illuminate\Routing\Route->run()
#9 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Routing\Router->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#10 /var/www/html/ninja5/app/Http/Middleware/SetInviteDb.php(53): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#11 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\SetInviteDb->handle(Object(Illuminate\Http\Request), Object(Closure))
#12 /var/www/html/ninja5/app/Http/Middleware/QueryLogging.php(38): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#13 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\QueryLogging->handle(Object(Illuminate\Http\Request), Object(Closure))
#14 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(41): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#15 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Routing\Middleware\SubstituteBindings->handle(Object(Illuminate\Http\Request), Object(Closure))
#16 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(77): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#17 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\VerifyCsrfToken->handle(Object(Illuminate\Http\Request), Object(Closure))
#18 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#19 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\View\Middleware\ShareErrorsFromSession->handle(Object(Illuminate\Http\Request), Object(Closure))
#20 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#21 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(63): Illuminate\Session\Middleware\StartSession->handleStatefulRequest(Object(Illuminate\Http\Request), Object(Illuminate\Session\Store), Object(Closure))
#22 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Session\Middleware\StartSession->handle(Object(Illuminate\Http\Request), Object(Closure))
#23 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#24 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse->handle(Object(Illuminate\Http\Request), Object(Closure))
#25 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#26 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Cookie\Middleware\EncryptCookies->handle(Object(Illuminate\Http\Request), Object(Closure))
#27 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#28 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Router.php(694): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#29 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Router.php(669): Illuminate\Routing\Router->runRouteWithinStack(Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request))
#30 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Router.php(635): Illuminate\Routing\Router->runRoute(Object(Illuminate\Http\Request), Object(Illuminate\Routing\Route))
#31 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Routing/Router.php(624): Illuminate\Routing\Router->dispatchToRoute(Object(Illuminate\Http\Request))
#32 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(166): Illuminate\Routing\Router->dispatch(Object(Illuminate\Http\Request))
#33 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}(Object(Illuminate\Http\Request))
#34 /var/www/html/ninja5/app/Http/Middleware/Cors.php(34): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#35 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\Cors->handle(Object(Illuminate\Http\Request), Object(Closure))
#36 /var/www/html/ninja5/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#37 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Fideloper\Proxy\TrustProxies->handle(Object(Illuminate\Http\Request), Object(Closure))
#38 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#39 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))
#40 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#41 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))
#42 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#43 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle(Object(Illuminate\Http\Request), Object(Closure))
#44 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(87): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#45 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance->handle(Object(Illuminate\Http\Request), Object(Closure))
#46 /var/www/html/ninja5/vendor/sentry/sentry-laravel/src/Sentry/Laravel/Tracing/Middleware.php(46): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#47 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Sentry\Laravel\Tracing\Middleware->handle(Object(Illuminate\Http\Request), Object(Closure))
#48 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#49 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(141): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#50 /var/www/html/ninja5/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(110): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter(Object(Illuminate\Http\Request))
#51 /var/www/html/ninja5/public/index.php(57): Illuminate\Foundation\Http\Kernel->handle(Object(Illuminate\Http\Request))
#52 {main}
"}

david · December 20, 2020, 9:07am

@gerootech this is a known issue with puppeteer. Sometimes it failed, however the way we process PDFs it will spool another job which usually completes, so it is safe to ignore this particular error message.

gerootech · December 20, 2020, 5:46pm

@david and @hillel thanks for your help to get this up and running, however few more things after we fixed the initial setup issues, Please advise if i need to create separate topic for each item or we can still work on this topic for troubleshooting:

Invoices getting displayed in the plain pdf template, not matter what template i choose in the settings.
The migration process didn’t migrate any of the expenses attachments.
I have a second user that was created as an admin, but when i used it to login i don’t see anything except Dashboard, Reports and Settings (only the user settings and device settings related to the user)
I still cannot edit the users, it displays a grey area when i try to edit - Correction only the user got migrated, i tried creating a new user and seems working fine…
I cannot use the mobile app (android) with http, it complains through the following error:

"Bad state: Insecure http is not allowed by platform:
http://in5.ghf.loc/api/v1/login?first_load=true&include_static=true

And still complains about https not trusted.

david · December 20, 2020, 11:42pm

@gerootech

We’ve had other users report this, the root cause is browser caching the PDF, @hillel we may need to consider cache busting this.
Expense Documents aren’t in the migration - yet .
@hillel should have a fix for this in the latest version?
I think there is a platform issue with self signed SSL certificates. I think the resolution here is to fall back to http, or use a lets encrypt certificate if you are on a public network.

gerootech · December 21, 2020, 3:49am

@david and @hillel for the PDF if you mean brwoser cache, i tried even in incognito mode, and didn’t help.

for the android client, i only use http for browser access, and it doesn’t work with http for the android mobile application as it used to work and i am getting the following error:

"Bad state: Insecure http is not allowed by platform:
http://in5.ghf.loc/api/v1/login?first_load=true&include_static=true

david · December 21, 2020, 4:09am

@gerootech yes the issue with the browser caching isn’t resolved even using an incognito window, there is a developer setting in chrome which needs to be disabled to stop the caching, i think we’ll need to bust the cache.

@hillel thoughts on android client not liking http:// routes?

gerootech · December 21, 2020, 4:11am

@david, thanks

one more question though, If i need to run another migration, do i need to reset the current v5 setup ??

david · December 21, 2020, 4:22am

The V4 side allows you to FORCE migrate companies, which will dump the old company on V5 and remigrate. If you don’t select FORCE, and the company is already detected the migration stops.

There is no way to do incremental migrations either.

hillel · December 21, 2020, 7:45am

Did you change the invoice design or the setting on Settings > Invoice Design?
I think this is a different issue, is the users showing as an admin on Settings > User Management

I believe by default Flutter apps prevent HTTP traffic, not sure how I feel about changing this.

gerootech · December 21, 2020, 12:11pm

@hillel

I did change setting on Settings > Invoice Design to use the Clean instead of plain. Also tried with others like business but nothing made any difference. Still getting invoices displayed in plain template.
It was working when i initially installed V5, but at some point stopped to work. Would you consider fixing the self signed certificate issue better than allowing http traffic?

hillel · December 21, 2020, 5:27pm

That setting only affects new invoices, you would need to change the design on the settings tab on the invoice edit page
We’ll consider it

gerootech · December 21, 2020, 6:02pm

@hillel,
so migration didn’t migrate the template used for the invoice. My invoices already were set to Clean on the V4?
2. Thanks for this other wise we don’t have access to the mobile apps at all.

hillel · December 21, 2020, 6:51pm

The v5 templates are built with HTML/CSS, the migration isn’t able to convert v4 templates.

gerootech · December 21, 2020, 6:55pm

@hillel, thanks for your quick response, is there a way to set this on the DB level ?? just to change all of them at one time instead of going to one at time ?

hillel · December 22, 2020, 5:36am

It may be possible but I don’t have the SQL query to share.

Note: in general we strongly advise against manually updating the database.

gerootech · December 22, 2020, 2:27pm

@hillel thanks, one more question though, if the template name is the same though why not to migrate the invoices with the name it is being used in v4?

For example i am using the clean template in v4 if we migraed the invoices to v5 with the same template name it will use the clean template but yhe html format that exists in v5, we won’t migrate the template itself.

Just a suggestion.

hillel · December 22, 2020, 3:30pm

Which design were the invoices set to?

@ben any thoughts on this?

ben · December 22, 2020, 3:45pm

Hey there. I believe default behaviour should be:

If you migrate invoice that uses “clean” on v4 it should be recreated on v5 using same template.

Isn’t that what’s happening right now?