Sebastian's Second 🧠

    Don't Panic, If Your Service Breaks Over Night

    [!INFO]-
    topic: 🖥️ Tech
    links:

    source: Experience
    tags: #permanent-note #published

    Last Modified: =dateformat(this.file.mtime, "MMM. dd, yyyy - HH:mm")

    If something is broken: Keep calm and analyze, before trying any quick fixes that break even more.

    Your internet is gone, your server is broken, your Nextcloud crashes.
    It may happen sporadically that one or all of your service do not work as expected.

    Keep calm and analyze, before trying any quick fixes from Google

    The worst thing you can do, is copy and paste any random command you don't even know the function of. You easily destroy more, than what was broken in the first place. Maybe the issue wasn't even on your end, but now it is.

    This has happened to me more than once, and it is super annoying.

    Try to remain calm and find out if the problem is your software, or something external, that you depend on.

    Follow these steps as a rule of thumb:

    1. Is your internet working? Reboot your router and whatever device is not working correctly
    2. If it is a self-hosted service, that is inaccessible from the outside anymore: how about from the inside? Can you still access it via its local IP? If so, does anything seem unusual from there?
      • If everything is fine from the inside, great! You only need to worry about how to access it from the outside again. If you are using feste-ip, consider waiting a few days. It already happened to me that it worked again later. Maybe set up the HTTPS-Proxy again (and if that fails, also wait. It worked again later).
      • If it doesn't go to the next step
    3. Try to carefully find the problem. Only read logs and find out how to access the logs. Do not try any quick fixes before you have an idea what the issue is.
    4. Maybe you have a recent backup to revert to? Consider using it before fiddling around. It is much less of a pain in the ass, and getting the data from a few days back might be much easier.
    5. Prefer steps that just install a missing dependency over fiddling with some configs. This is much less fatal.
    6. Always note, what you changed in the configs and make a backup file! Oh, how often I forget this and do not know what I have done if it worked (and I would like to know what fixed it) or it didn't work, and I don't know all the steps to revert…