Fresh v5 Docker install -- "500 | Server Error" and "production.INFO: db fails"

ShobuPrime · July 28, 2020, 11:34pm

Hello. This is my first time posting here, but I hope to provide some valuable insight to the community.

This post is going to attempt to serve both as an ask for help, and documentation for the steps I’ve taken to try and solve some issues.

I’ve been trying to get InvoiceNinja v5 to work in Docker for a while now. I made my own Dockerfile <shobuprime/invoiceninja:5> to fill in some of the holes the official one has for the latest build, such as missing “npm” and “node”. Other key differences include using php-fpm version 7.4.8 and the recently released Invoice Ninja 5.0.13

On the setup screen I’m able to generate a “Test PDF”, “Test Connection” to the database, and “Send a test email”. The trouble I have is, after filling out the user details and clicking “Submit”, I get a “500 | Server Error” page.

Initially, I noticed in “laravel.log”, the only errors I see are “production.INFO: db fails”

While writing this, I did see one more error pop up for the first time after hours of troubleshooting:
production.INFO: SQLSTATE[HY000] [2002] No such file or directory (SQL: select * from information_schema.tables where table_schema = ninja and table_name = migrations and table_type = ‘BASE TABLE’)

The mysql container I’m running is mysql:latest. I’ve tried 5 and 5.7 and still get the same results.

I’m using NGINX:1 as my web server, and it doesn’t give much helpful info in it’s error logs. I did receive a “405 Not Allowed NGINX error” after attempting to fill all appropriate information on the setup screen. It felt like a timeout, and the logs pointed to a timeout.

I notice the “500 | Server Error” would show when I did not fill in any email settings.

Now, while writing this, I did try a couple of other things:

In the URL, box, I was typing “https://url”

On my most recent attempt, silly me didn’t even to think of trying to type only URL in the box. It feels like that helped as well, as I’m finally able to access the login screen and sign in with my test account, but I still would at least appreciate some more verbose errors somewhere.

One other issue I noticed was that I’m only able to generate PDF’s with my invoiceninja image in privileged mode. I’ve already read about the “–no-sandbox” solutions with puppeteer and all that, but I’m wondering if someone could help point me in a direction to ensure all of these speedbumps I’ve faced while configuring this image can be ironed out in a better way during the image build, or if there’s a config file I can import.

While I did end up getting the product to technically work, I would like to know what your thoughts are on the matter. I’d be happy to post the docker-compose I’ve used, and the NGINX config if any one if curious.

Thanks for your time.

david · July 29, 2020, 12:43am

Hi @ShobuPrime i’m not an expert on Docker, however we would like to focus on an improved experience setting InvoiceNinja up with Docker.

I’ve had a quick look at your DockerFile, I can’t see how you are setting up your DB or persisting the mysql data anywhere?

As for the PDF issue, are you seeing a specific error we can work on?

ShobuPrime · July 29, 2020, 2:26am

Hey @david, thanks for reaching out. Prepare for the wall of text!

I can assure you the database data is persistent.

I’m currently running InvoiceNinja in the following docker compose:

version: “2”

services:
db:
image: mysql:8
restart: always
volumes:
- /invnj/db:/var/lib/mysql
environment:
- MYSQL_DATABASE=db-ninja-01
- MYSQL_PASSWORD=ninja
- MYSQL_USER=ninja
- MYSQL_ROOT_PASSWORD=ninjaAdminPassword
networks:
- default

app:
image: shobuprime/invoiceninja:5
#image: invoiceninja/invoiceninja:alpine-4
restart: always
depends_on:
- db
volumes:
- /invnj/storage:/var/www/app/storage
- /invnj/logo:/var/www/app/public/logo
- /invnj/public:/var/www/app/public
environment:
- APP_ENV=production
- INVOICENINJA_USER=invoiceninja
- LOG=errorlog
- DB_PASSWORD=ninja
- DB_USERNAME=ninja
- DB_DATABASE=db-ninja-01
- DB_HOST=db
- DB_STRICT=false
- DB_TYPE=mysql
- APP_CIPHER=AES-256-CBC
- APP_KEY=SomeRandom32bitString
- APP_URL=https://myurlhere
- APP_DEBUG=0
networks:
- default

web:
image: nginx:1
volumes:
- /invnj/config/nginx.conf:/etc/nginx/nginx.conf:ro
- /invnj/storage:/var/www/app/storage
- /invnj/logo:/var/www/app/public/logo
- /invnj/public:/var/www/app/public
- /invnj/ssl/fullchain.pem:/etc/nginx/ssl/fullchain.pem:ro
- /invnj/ssl/privkey.pem:/etc/nginx/ssl/privkey.pem:ro
- /invnj/ssl/dhparam.pem:/etc/nginx/ssl/dhparam.pem:ro
- /invnj/logs/nginx:/var/log/nginx/
expose:
- “80”
- “443”
depends_on:
- app
ports: # Delete if you want to use reverse proxy
- 443:443
networks:
- default

I do plan on creating a monolithic image with mysql and web server built in eventually, but that’s the setup for now.

As for the PDF’s, here are the error’s I’m getting.

This first one happens on the setup page, and if I don’t run the container as privileged:

[2020-07-29 00:24:26] production.INFO: The command “PATH=$PATH:/usr/local/bin NODE_PATH=npm root -g node ‘/var/www/app/vendor/spatie/browsershot/src/…/bin/browser.js’ '{“url”:“file:///tmp/62987628-0972814001595982262/index.html”,“action”:“pdf”,“options”:{“path”:”/var/www/app/public/test.pdf",“args”:[],“viewport”:{“width”:800,“height”:600},“displayHeaderFooter”:false}}’" failed.

Exit Code: 1(General error)

Working directory: /var/www/app/public

Output:

Error Output:

Error: Failed to launch chrome!
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
[0729/002426.610929:FATAL:zygote_host_impl_linux.cc(190)] Check failed: ReceiveFixedMessage(fds[0], kZygoteBootMessage, sizeof(kZygoteBootMessage), &boot_pid).
Received signal 6
r8: 00007ffcec0c78f4 r9: 00007fedfd036a80 r10: 0000000000000008 r11: 0000000000000246
r12: 00007ffcec0c80c0 r13: 00007ffcec0c7ffc r14: 00007ffcec0c7e58 r15: 00000000000000a0
di: 0000000000000002 si: 00007ffcec0c77c0 bp: 00007ffcec0c77c0 bx: 0000000000000000
dx: 0000000000000000 ax: 0000000000000000 cx: 00007fedfdb73a71 sp: 00007ffcec0c77b8
ip: 00007fedfdb73a71 efl: 0000000000000246 cgf: aaaa000000000033 erf: 0000000000000000
trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

at onClose (/var/www/app/node_modules/puppeteer/lib/Launcher.js:349:14)
at ChildProcess.<anonymous> (/var/www/app/node_modules/puppeteer/lib/Launcher.js:339:60)
at ChildProcess.emit (events.js:327:22)
at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)

This next one is when I’m logged in and try to preview a PDF. Doesn’t matter if the container is privileged or not.

[2020-07-29 00:36:06] production.ERROR: The command “PATH=$PATH:/usr/local/bin NODE_PATH=/usr/bin /usr/bin root -g /usr/bin ‘/var/www/app/vendor/spatie/browsershot/src/…/bin/browser.js’ ‘{“url”:“file:///tmp/210917052-0000350001595982966/index.html”,“action”:“pdf”,“options”:{“args”:[],“viewport”:{“width”:800,“height”:600,“deviceScaleFactor”:1},“displayHeaderFooter”:false,“waitUntil”:“networkidle0”,“printBackground”:true}}’” failed.

Exit Code: 126(Invoked command cannot execute)

Working directory: /var/www/app/public

Output:

Error Output:

sh: /usr/bin: Permission denied
sh: /usr/bin: Permission denied
{“userId”:1,“exception”:"[object] (Symfony\Component\Process\Exception\ProcessFailedException(code: 0): The command “PATH=$PATH:/usr/local/bin NODE_PATH=/usr/bin /usr/bin root -g /usr/bin ‘/var/www/app/vendor/spatie/browsershot/src/…/bin/browser.js’ ‘{“url”:“file:\/\/\/tmp\/210917052-0000350001595982966\/index.html”,“action”:“pdf”,“options”:{“args”:[],“viewport”:{“width”:800,“height”:600,“deviceScaleFactor”:1},“displayHeaderFooter”:false,“waitUntil”:“networkidle0”,“printBackground”:true}}’” failed.

Exit Code: 126(Invoked command cannot execute)

Working directory: /var/www/app/public

Output:

Error Output:

sh: /usr/bin: Permission denied
sh: /usr/bin: Permission denied
at /var/www/app/vendor/spatie/browsershot/src/Browsershot.php:738)
[stacktrace]
#0 /var/www/app/vendor/spatie/browsershot/src/Browsershot.php(532): Spatie\Browsershot\Browsershot->callBrowser(Array)
#1 /var/www/app/app/Utils/Traits/Pdf/PdfMaker.php(43): Spatie\Browsershot\Browsershot->pdf()
#2 /var/www/app/app/Jobs/Util/PreviewPdf.php(61): App\Jobs\Util\PreviewPdf->makePdf(NULL, NULL, ‘…’)
#3 [internal function]: App\Jobs\Util\PreviewPdf->handle()
#4 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(32): call_user_func_array(Array, Array)
#5 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Util.php(37): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#6 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(90): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#7 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(34): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))
#8 /var/www/app/vendor/laravel/framework/src/Illuminate/Container/Container.php(590): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)
#9 /var/www/app/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(94): Illuminate\Container\Container->call(Array)
#10 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(130): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}(Object(App\Jobs\Util\PreviewPdf))
#11 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(105): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(App\Jobs\Util\PreviewPdf))
#12 /var/www/app/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(98): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#13 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Bus/Dispatchable.php(26): Illuminate\Bus\Dispatcher->dispatchNow(Object(App\Jobs\Util\PreviewPdf))
#14 /var/www/app/app/Http/Controllers/PreviewController.php(166): App\Jobs\Util\PreviewPdf::dispatchNow(’…’, Object(App\Models\Company))
#15 /var/www/app/app/Http/Controllers/PreviewController.php(126): App\Http\Controllers\PreviewController->blankEntity()
#16 [internal function]: App\Http\Controllers\PreviewController->show()
#17 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): call_user_func_array(Array, Array)
#18 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\Routing\Controller->callAction(‘show’, Array)
#19 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Route.php(219): Illuminate\Routing\ControllerDispatcher->dispatch(Object(Illuminate\Routing\Route), Object(App\Http\Controllers\PreviewController), ‘show’)
#20 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Route.php(176): Illuminate\Routing\Route->runController()
#21 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Router.php(681): Illuminate\Routing\Route->run()
#22 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(130): Illuminate\Routing\Router->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request))
#23 /var/www/app/app/Http/Middleware/Locale.php(34): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#24 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): App\Http\Middleware\Locale->handle(Object(Illuminate\Http\Request), Object(Closure))
#25 /var/www/app/app/Http/Middleware/TokenAuth.php(81): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#26 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): App\Http\Middleware\TokenAuth->handle(Object(Illuminate\Http\Request), Object(Closure))
#27 /var/www/app/app/Http/Middleware/SetDb.php(41): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#28 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): App\Http\Middleware\SetDb->handle(Object(Illuminate\Http\Request), Object(Closure))
#29 /var/www/app/app/Http/Middleware/StartupCheck.php(68): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#30 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): App\Http\Middleware\StartupCheck->handle(Object(Illuminate\Http\Request), Object(Closure))
#31 /var/www/app/app/Http/Middleware/QueryLogging.php(40): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#32 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): App\Http\Middleware\QueryLogging->handle(Object(Illuminate\Http\Request), Object(Closure))
#33 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(41): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#34 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Illuminate\Routing\Middleware\SubstituteBindings->handle(Object(Illuminate\Http\Request), Object(Closure))
#35 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(59): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#36 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Illuminate\Routing\Middleware\ThrottleRequests->handle(Object(Illuminate\Http\Request), Object(Closure), 60, ‘1’)
#37 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(105): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#38 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Router.php(683): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#39 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Router.php(658): Illuminate\Routing\Router->runRouteWithinStack(Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request))
#40 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Router.php(624): Illuminate\Routing\Router->runRoute(Object(Illuminate\Http\Request), Object(Illuminate\Routing\Route))
#41 /var/www/app/vendor/laravel/framework/src/Illuminate/Routing/Router.php(613): Illuminate\Routing\Router->dispatchToRoute(Object(Illuminate\Http\Request))
#42 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(170): Illuminate\Routing\Router->dispatch(Object(Illuminate\Http\Request))
#43 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(130): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}(Object(Illuminate\Http\Request))
#44 /var/www/app/app/Http/Middleware/Cors.php(35): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#45 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): App\Http\Middleware\Cors->handle(Object(Illuminate\Http\Request), Object(Closure))
#46 /var/www/app/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#47 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Fideloper\Proxy\TrustProxies->handle(Object(Illuminate\Http\Request), Object(Closure))
#48 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#49 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))
#50 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#51 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle(Object(Illuminate\Http\Request), Object(Closure))
#52 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#53 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle(Object(Illuminate\Http\Request), Object(Closure))
#54 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/CheckForMaintenanceMode.php(63): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#55 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(171): Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode->handle(Object(Illuminate\Http\Request), Object(Closure))
#56 /var/www/app/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(105): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#57 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(145): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#58 /var/www/app/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(110): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter(Object(Illuminate\Http\Request))
#59 /var/www/app/public/index.php(57): Illuminate\Foundation\Http\Kernel->handle(Object(Illuminate\Http\Request))
#60 {main}
"}

david · July 29, 2020, 5:20am

Ok, i was looking at the DockerFile not the docker composse file, so your MySQL setup makes sense.

One trick for running puppeteer is to add this options to allow it to run in the container

RUN echo 'kernel.unprivileged_userns_clone=1' > /etc/sysctl.d/userns.conf

However, make sure to start your container with --cap-add=SYS_ADMIN , e.g.

docker run -it --init --rm --cap-add=SYS_ADMIN --name puppeteer-chrome-debian_c puppeteer-chrome-debian_i node -e " cat screenshot.js "

This may or may not solve the next problem you are seeing. Let us know if this makes a difference.

ShobuPrime · July 29, 2020, 3:43pm

@david, thanks so much for your input.

I’ve tried the RUN command and the SYS_ADMIN flag. The image will build with the RUN command, but the SYS_ADMIN flag crashes the container.

If I execute the container from the build with the successful RUN command and no SYS_ADMIN flag, I’m still unable to generate PDFs.

If I have some time, I could try to build a debian image instead of alpine to see if this works better, because I did notice some comments mentiong the command you provided was for debian.

Running with the --privileged flag is still the only method I got to work so far regarding PDFs, but obviously for production this is not practical as I run other services on this server.

Now, I should also mention I also commented out 2 environment variables from the Dockerfile you saw:

ENV NODE_PATH=/usr/bin
ENV NPM_PATH=/usr/bin

This fixed the errors I was getting regarding permission denied, which is a step in the positive direction.

There are other quirks which I’m trying to work out, such as an occasional 405 nginx error after setup, and setup redirecting to the setup page after filling out all of the info. However,I think I’ve invested too much personal time into this already.

I’m able to get up and running much faster with your official alpine-4 image and downgrading the mysql container from 8 fo 5 while keeping the existing NGINX container with the same exact config.

Compared to the work I was doing before, everything “just works”, except for emails. Test email during setup works, but I’m receiving “fwrite(): SSL: Connection reset by peer” and I notice the settings page doesn’t keep the username and password, which looks like the cause.

Please feel free to take my existing master dockerfile for 5 and incorporate it into your official one for version 5, since at least I managed to build successfully and technically get everything working with all dependencies, including node and npm. Your official one is missing a lot.

I feel instead of using 5 for production, I’m going to just play around with it until it gets to a stable enough experience. I also notice not all features from 4 exist in 5 yet anyways.

david · July 29, 2020, 10:19pm

@ShobuPrime

Thanks for your help with this! It sounds like we are really close. I’ll dedicate some time to this so we can get this over the line and merge this work into the official repo