Added my repo script to the shed.

README.md

@@ -3,9 +3,17 @@
 
 For me, a shed is a place to put build, learn and ideally make something useful.
 
-This git repository is a "fit for purpose" interpretation of a shed. Our community routinely discuss our shared interests in person and online. We do our best to demonstrate our personal projects in person, or with sni
-about lots of shared interests, but have struggled to find a way to share materials that help us 
+This git repository is a "fit for purpose" interpretation of a shed. Our community routinely discuss our shared interests in person and online. We do our best to demonstrate our personal projects, either by way of demonstration or with code snippets pasted in forum posting. We now have a place to publish demonstrations that we can compliment with a discussion on our Discourse forum.
 
-This one happens to be a complement to [Homelab Brisbane](https://homelabbrisbane.com.au/)'s community. You're probably reading this because you're involved in a conversation with another community member and you want to complement your conversation with some practical collaboration. Maybe you're talking about an `ansible` playbook, a `bash` script or a `docker-compose` configuration.
+# Folder Layout
+
+The loose convention is that the first level folder names correspond to the Discourse handle of the owner. My contributions for example are under `jdownie`. Under that will be a folder for each area of interest. For example, I have written for myself a script that helps me work in numerous `git` repositories across multiple machines. That script is called `repo`, so i'm hosting it in the `jdownie/repo` folder.
+
+That `repo` folder for example, not only contains the script itself, but a `README.md` to explain it in more detail and an example configuration file.
+
+# How and Why I moved `repo` to the Shed
+
+I once hosted and maintained that script in my own personally hosted repository. I have "moved it into the shed" so that others can either use it directly, or copy and experiment on their own copy. Another community member, `tdurden` might take interest in that script. They'd create `tdurden/repo` and clone `jdownie/repo` into it. From this point, `tdurden` might make improvements and discuss them with `jdownie` on Discourse. `jdownie` might like to adopt those improvements, and with `diff` and a little care, migrate those changes back into `jdownie/repo`. Alternatively, `tdurden` might break their copy, and ask `jdownie` to take a look in the shed to help out.
+
+I'm using `repo` as an illustration of what i hope we can accomplish with this "shed" idea. Others might start to maintain their `ansible` playbooks or `docker-compose.yaml` files for example. It's really just a shared folder. `git` makes it a bit more structured and transactional than a `Dropbox` folder (for example).
 
-The loose convention is that the first level folder names correspond

jdownie/repo/README.md

@@ -0,0 +1,34 @@
+
+# How I use Git
+
+I host a few `git` repositories on my own personal `gitea` instance. The ones I use the most are...
+
+- `bin`, for my scripts (including my `.bash_profile` script
+- `cfg`, for my configuration files
+- `Notes`, or my Obsidian notes
+- `Orchestration`, for my container and virtual machine solutions
+
+There are also a few projects that I build myself...
+
+- `neovim`
+- `multibg-wayland`
+
+Anyway, there's lots more repositories and not all of them are required on all of my machines. While i'm working on a machine i'll make changes in oneor many repositories, so before I shut that machine down i need to `git add`, `git commit` and `git push`.
+
+# How This Script Helps
+
+In my `.bash_profile` script i set the environment variable `REPO_CFG` to point to my `repo.yaml` file (which is in my `cfg` repository by the way). I also add this `jdownie/repo` folder to my `PATH`. There is an example of a `repo.yaml` in this folder. For each repository, there's a code, a url and a local path to clone the repository into. There's also a list of hostnames that the repository is wanted on (which can be an empty list if the repository is wanted on all hosts; like `bin` and `cfg` for example).
+
+With all of that in place, i can run the following commands across all of my repositories...
+
+- `repo status`, list each repo with a count of "dirty" files against each
+- `repo lc`, stage and commit all files in each repo with a generic comment
+- `repo fetch`, `repo pull` and `repo push`
+- `repo sync`, pulls and pushes all repositories
+
+This script makes it easy for me to run `repo lc` and `repo sync` before i shut a machine down. On my next machine I can run `repo sync` to get my changes on the new machine.
+
+If I want to remove a repo from a host. I can remove the hostname from that repositorie's `hosts` list. Then i run `repo prune` which does a `lc`, a `sync` and then removes the cloned folder.
+
+Alternatively, i might move a repository. I change the url in `repo.yaml`, and then run `repo align` to update that repository's remote url.
+

jdownie/repo/repo

@@ -0,0 +1,171 @@
+#!/usr/bin/env python
+
+import socket
+import yaml, sys, subprocess, os
+import concurrent.futures
+import re
+import shutil
+
+def hostname():
+  ret = socket.gethostname().split('.', 1)[0]
+  return ret
+
+def parse_yaml(file_path):
+  with open(file_path, 'r') as file:
+    try:
+      data = yaml.safe_load(file)
+      return data
+    except e:
+      print(f"Error parsing YAML file: {e}")
+      return None
+
+def execute_command(command, dump_error = True):
+  command = f"/bin/bash -c '{command}'"
+  try:
+    result = subprocess.run(command, shell=True, check=True, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if result.returncode == 0:
+      return result.stdout.strip()
+    else:
+      print(f"Error executing command: {result.stderr.strip()}")
+      print(command)
+      return None
+  except subprocess.CalledProcessError as e:
+    if dump_error:
+      print(f"Error executing command: {e.stderr.strip()}")
+      print(command)
+    return None
+
+def perform_action(action, key, item, silent = False):
+  output = None
+  lbl = "{0}ing".format(action).title()
+  if os.path.exists(item["path"]):
+    if action in list([ "pull", "push", "fetch" ]):
+      push = True
+      if "push" in item.keys():
+        push = item["push"]
+      if push or action in list([ "pull", "fetch" ]):
+        cmd = "git -C \"{0}\" {1}".format(item["path"], action)
+        if not silent:
+          print("{0} {1}...".format(lbl, key))
+        output = execute_command(cmd)
+    elif action == "sync":
+      if not silent:
+        print("{0} {1}...".format(lbl, key))
+      perform_action("pull", key, item, silent=True)
+      perform_action("push", key, item, silent=True)
+    elif action == "lcs":
+      if not silent:
+        print("{0} {1}...".format(lbl, key))
+      perform_action("pull", key, item, silent=True)
+      perform_action("lc", key, item, silent=True)
+      perform_action("pull", key, item, silent=True)
+      perform_action("push", key, item, silent=True)
+    elif action == "lc":
+      cmd = "git -C \"{0}\" status --porcelain".format(item["path"])
+      output = execute_command(cmd).split("\n")
+      if len(output[0]) > 0:
+        print("Lazy committing {0}...".format(key))
+        cmd = "git -C \"{0}\" add .".format(item["path"])
+        output = execute_command(cmd)
+        cmd = "git -C \"{0}\" commit -m \"Lazy commit on {1}.\"".format(item["path"], hostname)
+        output = execute_command(cmd)
+  return output
+
+if __name__ == "__main__":
+  yaml_file_path = os.getenv("REPO_CFG")
+  if not os.path.exists(yaml_file_path):
+    print(f"Environment variable REPO_CFG needs to point to your repo.yaml file.", file=sys.stderr)
+    sys.exit(1)
+  try:
+    cfg = parse_yaml(yaml_file_path)
+  except:
+    print(f"Unable to parse {yaml_file_path}.", file=sys.stderr)
+    sys.exit(2)
+  r = list(cfg.keys())
+  # Let's quickly sweep through and expand any tildes ("~") in the path references...
+  for k in r:
+    path = cfg[k]["path"]
+    path = os.path.expanduser(path)
+    cmd = f"git config --global --add safe.directory {path}"
+    # I am not outputting any errors here because of a dumb bug in WSL.
+    output = execute_command(cmd, False)
+    cfg[k]["path"] = path
+  if len(sys.argv) == 3:
+    if not sys.argv[2] in cfg.keys():
+      print("{0} is not one of your repositories.".format(sys.argv[2]))
+      exit(1)
+    r = list([ sys.argv[2] ])
+  if sys.argv[1] == "list":
+    for k in cfg.keys():
+      print(k)
+  elif sys.argv[1] == "status":
+    for k in r:
+      if os.path.exists(cfg[k]["path"]):
+        cmd = "git -C \"{0}\" status --porcelain".format(cfg[k]["path"])
+        output = execute_command(cmd).split("\n")
+        status = "-"
+        if len(output[0]) > 0:
+          status = len(output)
+        print("{0} {1}".format(str(status).rjust(3), k))
+  elif sys.argv[1] in list( [ "sync", "lc", "pull", "push", "fetch" ] ):
+    thread_count = 10
+    with concurrent.futures.ThreadPoolExecutor(max_workers=thread_count) as executor:
+      futures = {executor.submit(perform_action, sys.argv[1], k, cfg[k]) for k in r}
+      for future in concurrent.futures.as_completed(futures):
+        future.result()
+        try:
+          future.result()  # To get the result of sync if it returns something
+        except Exception as exc:
+          print(f"{exc}")
+    # for k in r:
+    #   perform_action(sys.argv[1], k, cfg[k])
+  elif sys.argv[1] == "clone":
+    hn = hostname()
+    for k in r:
+      hosttest = True
+      if "hosts" in cfg[k].keys():
+        hosttest = hn in cfg[k]["hosts"]
+      if hosttest and not os.path.exists(cfg[k]["path"]):
+        print("Cloning {0} into {1}...".format(k, cfg[k]["path"]))
+        cmd = "git clone \"{0}\" \"{1}\"".format(cfg[k]["url"], cfg[k]["path"])
+        output = execute_command(cmd)
+  elif sys.argv[1] == "align":
+    hn = hostname()
+    p = re.compile("^.*Fetch URL: (.*)$")
+    for k in r:
+      hosttest = True
+      if "hosts" in cfg[k].keys():
+        hosttest = hn in cfg[k]["hosts"]
+      if hosttest and os.path.exists(cfg[k]["path"]):
+        print("Aligning {0}...".format(k))
+        cmd = "git -C \"{0}\" remote show -n origin".format(cfg[k]["path"])
+        output = execute_command(cmd)
+        url = None
+        for line in output.split("\n"):
+          r = p.match(line)
+          if r != None:
+            url = r.group(1)
+        if url == None:
+          print("Unable to determine origin's remote path.")
+        else:
+          if cfg[k]["url"] == url:
+            print("  🟢 {0}".format(url))
+          else:
+            print("  🔴 {0}".format(url))
+            cmd = "git -C \"{0}\" remote set-url origin \"{1}\"".format(cfg[k]["path"], cfg[k]["url"])
+            output = execute_command(cmd)
+            print("  🟢 {0}".format(cfg[k]["url"]))
+      # else:
+      #   print("Failed hosttest for {0}".format(k))
+  elif sys.argv[1] == "prune":
+    hn = hostname()
+    for k in r:
+      hosttest = True
+      if "hosts" in cfg[k].keys():
+        hosttest = hn in cfg[k]["hosts"]
+      if not hosttest and os.path.exists(cfg[k]["path"]):
+        perform_action("lc", k, cfg[k])
+        perform_action("sync", k, cfg[k])
+        print("Pruning {0}".format(k))
+        shutil.rmtree(cfg[k]["path"])
+

jdownie/repo/repo.yaml

@@ -0,0 +1,13 @@
+neovim:
+  path: ~/Build/neovim
+  url: https://github.com/neovim/neovim.git
+  push: false
+  hosts: []
+hblShed:
+  path: ~/Development/HLB/shed
+  url: ssh://git@gitea.downie.net.au:32222/jdownie/shed.git
+  hosts:
+   - fry
+   - yancy
+   - frankie
+   - scruffy