Database *POOF* reminder to backup!

I’m not even really sure what happened this afternoon but if anyone wants as story here goes:

Environment is a windows mini PC (I know…) running docker with IN. I wanted to physically move the PC today so I very nicely shut down all containers before shutting down the computer. Upon bringing it back up (all good) I wanted to test IN as we are about to send our first actual invoice out and go live this week. I open up IN and login and BOOM “Please enter company name” I’m like what!? Everything is wiped out to stock. I look at the container logs and literally see where it re-seeded the DB. I still don’t understand why (or where). I’m freaking out thinking something really broken. I had taken a manual DB backup just after I got everything working. I’m looking at my files and everything is still there DB doesn’t appear to have been changed with today’s date but I’m still preparing for a restore… Still trying to wrap my brain around what could be wrong I’m starting to think my WSL instance is borked and IN “moved” to a different instance but I can’t find evidence of it anywhere.

I try rebuilding the containers, same issue. I’m like what is going on… Finally I deleted the DB container and then rebuilt again. It’s back!!! All my client entries, products, templates etc. OMG wew! I’m still a bit baffled where this “other” database was running from while it was messed up. This "other instance survived a full host reboot and container rebuilds until I fully deleted it and forced a new creation.

Wild. I’m still pretty new to containers but each time something goes wrong (or I screw it up) I like the concept more and more lol. Easy to get back. I do think I need to work on some automated backups however!

Something with persistent storage I bet.

So this happened again. Automatic updates ran on the host and it rebooted overnight - disabled. But this morning I again had full delete the DB container and issue a docker command to rebuild and then it’s fine again pointing at my existing database. Not really sure why on host reboot it’s creating a new or looking at some other DB. The params look the same to me and I couldn’t actually find the new DB files it defaults to on reboot.

Really not sure why this keeps happening but just to continue chronicling:

I needed to reboot the host again this weekend for updates and I took these steps… I shut down all three docker docker containers. I actually changed “restart always” on the APP and SERVER containers to never thinking maybe they are starting up before the DB container. Rebooted host, and verified that only DB container had auto restarted. Started up the SERVER container first and then the APP container.

The result was that another phantom database was still created. I still have no idea where it is being created but it must not be saving as I can watch it being built in the APP log. I had to again delete the DB container and issue a docker compose up command to get it to rebuild and then its pointing at my actual DB again.

In all honesty, and with all the experience that I have, I would never recommend running a database in a Docker container. A database is quite complex as it is, so my recommendation would be to simply install a database straight on your host itself or install it on another machine. It’s one of the most complex things you can run in containers.

I work quite a lot with Kubernetes and it was preferred for the longest time to just run your database outside of it.

Running your database in a container just adds another layer that can complicate or break things. I’m not saying you can’t do it properly, just that it becomes easier to break.

In order to investigate what is going wrong in your case, you would need to provide the docker-compose files. I would however just suggest, especially if you are new with containers, to just install the database on your host directly.

https://medium.com/@bm54cloud/persistent-data-and-docker-ac12e2a00ee5

I’ll admit I’m new to containers but my data IS persistent. As I stated in my post this issue only occurs on a host reboot and is recoverable (with all my pre-existing data) if I delete the DB container from this stack and recreate it.