Textpattern CMS support forum
You are not logged in. Register | Login | Help
- Topics: Active | Unanswered
Github down, shards lost, repos rolled back
I can’t find news about it yet, but seeing reports in M’don: apparently GitHub was down for several nearly three hours while I was sleeping and ‘shards’, whatever those are, were lost, resulting in many repositories getting rolled back as far as three years without ability to recover changes since. That doesn’t sound good.
I hope Txp resources are okay. A quick look at docs appears nothing wrong there. Whew! ;)
That is certainly an argument for self-hosting.
But probably not feasible for projects like Txp without a foundation to coordinate by, or whatever. Otherwise a lot of trust involved, I guess.
Last edited by Destry (2018-10-22 09:12:42)
Offline
Re: Github down, shards lost, repos rolled back
Github is still glitching, my today commit has gone west.
Offline
Re: Github down, shards lost, repos rolled back
Offline
Re: Github down, shards lost, repos rolled back
Destry wrote #314682:
I hope Txp resources are okay. A quick look at docs appears nothing wrong there. Whew! ;)
That is certainly an argument for self-hosting.
But probably not feasible for projects like Txp without a foundation to coordinate by, or whatever.
Well, possibly possible in the grand scheme of things. When the significant job of migrating sites away from Joyent, sorting the snagging and then improving security and performance further is done and dusted, I would like to have the docs site moved to our own hosting now we can support it on DigitalOcean.
That would mean that GitHub would handle the version control and raw files, and then the tech that currently makes the docs site work with a Textpattern branded wrapper could be hosted on our servers instead. Same service, same appearance, same content, different server.
…but then that means if our server goes down, so do the docs. It’s a bit of a tightrope.
Offline
Re: Github down, shards lost, repos rolled back
Re: Github down… Welcome to microsoft
Re: txp development: gogs.
Re: site backups. I’m sure that between us we will organise to have at least one backup a week.
Yiannis
——————————
NeMe | hblack.art | EMAP | A Sea change | Toolkit of Care
I do my best editing after I click on the submit button.
Offline
Re: Github down, shards lost, repos rolled back
colak wrote #314695:
Re: Github down… Welcome to microsoft
That was my first thought but Microsoft only got approved by the EU last week so I doubt the takeover has been completed yet.
It is embarrassing, though.
Offline
Re: Github down, shards lost, repos rolled back
colak wrote #314695:
Re: Github down… Welcome to microsoft
Microsoft hasn’t actually acquired GitHub yet, regulatory approval was ongoing when I last checked a week ago.
Offline
Re: Github down, shards lost, repos rolled back
Re: site backups. I’m sure that between us we will organise to have at least one backup a week.
I would have thought that git, being a distributed workflow model means that there are always multiple copies of a repository at any one time. Even if not all are up to date, at least one must be.
TXP Builders – finely-crafted code, design and txp
Offline
Re: Github down, shards lost, repos rolled back
colak wrote #314695:
Re: txp development: gogs.
Hadn’t seen that one before. I was about to say Gitea is another one that looks very similar – in fact they look so similar (even in website wording) I wondered if one is a knock-off of the other. It seems that gitea is a fork of gogs. Both require “Go” as programming language, which isn’t on your everyday host…
TXP Builders – finely-crafted code, design and txp
Offline
Re: Github down, shards lost, repos rolled back
jakob wrote #314699:
a distributed workflow model means that there are always multiple copies of a repository at any one time.
Exactly. As long as someone has a pre-crash copy and hasn’t done a git pull since the problem was discovered, there’ll be at least one local copy from any of the devs or testers in the community to rebuild from.
The only unique things that GitHub house on our behalf are:
- Releases (we just have tags in the code where releases are made).
- Docs (but if someone has a local clone of the docs repo then we can rebuild from that if necessary).
- Issue queues.
So the only loss – as far as I can tell – if GitHub goes south like this are Release notifications and Issue queues. I’ve no idea if either of those are backupable or if there’s a way to at least make those distributed in some way to move towards satisfying Stef’s law of data survivity.
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline
Re: Github down, shards lost, repos rolled back
I would absolutely not condone moving off GitHub to a self-hosted git solution. To do so would be effectively kill our project in any meaningful sense.
While I’m sympathetic to the calls on this forum to give power back to the people and away from the Silicon Valley big hitters*, I also live in the real world; where project visibility is the difference between a dead project and a one with a future. GitHub rose to the top because it was (and still is) the best integration of version control for collaborative and social coding. Just because it potentially will be owned by Microsoft is moot to me.
*For transparency: Amazon are one of my clients.
PS. I daily pull any changes to all the Textpattern repos down to my local machine with git so at worst if there is a massive outage at GitHub we would only be max 24 hours of work lost.
PPS. There may be a way to host the docs site on our own servers, but the process would be:
- Install a Jekyll instance on our servers – that would mean having Ruby available plus about a hundred or so third-party gem dependencies that Jekyll needs to run – each gem with the potential to open security holes.
- Fetch the repo every time a commit is pushed to GitHub.
- Run a Jekyll build after fetch.
Offline
Re: Github down, shards lost, repos rolled back
I’m not advocating a move in any way. This outage just highlights where we have all our eggs in one basket and I was exploring ways to ensure we have legitimate backups of such content in case GitHub goes south permanently.
Losing the issue queue or release notifications would be a major bummer, so if there’s some way to back them up periodically to preserve the history, that’d be ace. They offer their own paid solution of course, but I’d prefer some other free alternative. There are some exporters or the API to roll our own but it’s unclear if it can get attachments and stuff like that.
The rest of the content on GitHub is all distributed anyway as far as I can tell so can always be migrated to some other central hub if the need ever arises.
Last edited by Bloke (2018-10-23 08:47:56)
The smd plugin menagerie — for when you need one more gribble of power from Textpattern. Bleeding-edge code available on GitHub.
Txp Builders – finely-crafted code, design and Txp
Offline